What’s as²t?

as²t is a family of hybrid LLMs of open models’ machine learning that features access to the model weights with derived capabilities freely available to individuals,...

How it works?

as²t helps you to write, plan, learn and… The simple answer: It responds using the public information it already knows publicly or fetches from other sources, like...

Hands-on access and community

For those eager to explore the capabilities of as²t, Bodhint has made it accessible via as²t AI Messaging Platform...

What is sparsity?

Sparsity uses the idea of conditional computation. While in dense models all the parameters are used for all the inputs, sparsity allows us to only run some parts of...

MoEs and Transformers

Transformers are a very clear case that scaling up the number of parameters improves the performance, so it’s not surprising that Google also explored this...

What does an expert learn?

The ST-MoE authors observed that encoder experts specialize in a group of tokens or shallow concepts. For example, we might end with a punctuation expert...