Skip to content

A TensorFlow C++ extension implementing a piecewise-linear approximation of sigmoid activation function used in Beatmup.

Notifications You must be signed in to change notification settings

lnstadrum/sigmoid_like_tf_op

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This repository contains a TensorFlow extension implementing a piecewise-linear approximation of sigmoid activation function used in Beatmup.

  • This function shrinks the activation signal into 0..1 range, enabling efficient fixed-point storage and reducing the memory bandwidth in inference.
  • Compared to the original sigmoid, this function is very efficient to compute in both forward and backward senses and does not suffer from vanishing gradient issue.

This activation function is featured in a dog image classifier running on Raspberry Pi GPU.

Activation functions

About

A TensorFlow C++ extension implementing a piecewise-linear approximation of sigmoid activation function used in Beatmup.

Topics

Resources

Stars

Watchers

Forks