x / (1 + tl.exp(-x)) 实现 silu | x * tl.sigmoid(x) | 128% |
tl.fdiv(x, y) / tl.div_rn(x, y) / x*(1.0/y) 实现 div | x / y(A5 fp32 上直接除最快) | 87%-94% |
x * tl.rsqrt(x) 实现 sqrt | tl.sqrt(x) | 71% |
tl.sqrt_rn(x) | tl.sqrt(x) | 55% |
tl.where(x < y, x, y) 实现 minimum | tl.minimum(x, y) | 75% |
tl.where(x > y, x, y) 实现 maximum | tl.maximum(x, y) | 53% |
tl.where(x>=0, x, -x) 实现 abs | tl.maximum(x, -x) 或 tl.abs(x) | 32% |
1.0/tl.rsqrt(x) 实现 sqrt | tl.sqrt(x) | 27% |
exp(x)/(1+exp(x)) 实现 sigmoid | 0.5*(1+tanh(x/2)) 或 tl.sigmoid(x) | 23% |
tl.exp2(x*LOG2E) 实现 exp | tl.exp(x) | 11% |
tl.log2(x) | tl.log(x) * LOG2E | 10% |