IP 검색

Fixed-point Transformer Self-attention Accelerate unit (BERT) (차세대지능형반도체사업단)

· Transformer encoder 모델인 BERT에 대해 high-accuracy inference가 가능한 self-attention module

Feature

· Precision: 16bit INT, fixed-point unit
· 높은 model accuracy 유지 (low accuracy drop of 0.1 or less for GLUE tasks)
· 높은 정확도 유지를 위한 internal mixed precision (upto 32bit)
· self-attention의 matrix multiplication 연산을 위한 2 output stationary systolic arrays (per single head core)
· output stationary systolic arrarys의 streaming input/output을 매 cycle마다 produce/comsume 할 수 있는, pipelined softmax module
· CORDIC algorithm-based exponential unit (using less BRAM than LUT-based)
· 기존의 CORDIC algorithm의 limited output range (-1~1) 문제를 해결하기 위한 extended CORDIC 적용
· Freq : 215MHz

Application

· SoC

Business Area

Accelerator

Category

Arithmetic & Mathematic > Arithmetic & Logic Unit

Tech Specs

IP Name :
Fixed-point Transformer Self-attention Accelerate unit (BERT) (차세대지능형반도체사업단)
Provider :
Hanyang Univeristy
FPGA Device :
Xilinx ZCU111
Foundry :
SAMSUNG
Technology :
12nm,16nm

Deliverables

· SW integration guide

Validation Status

· RTL simulation-based validation, FPGA-based validation, SW modeling & model accuracy evaluation

Availability

Available

Functional Diagram

Benefits

· -

List

POPUP ZONE

PIM IP

IP 검색