Lil'Log

Why We Think

3 months ago

Special thanks to John Schulman for a lot of super valuable feedback...

Reward Hacking in Reinforcement Learning

8 months ago

Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or...

Extrinsic Hallucinations in LLMs

about 1 year ago

Hallucination in large language models usually refers to the model generating unfaithful...

Diffusion Models for Video Generation

over 1 year ago

Diffusion models have demonstrated strong results on image synthesis in past years...

Prompt Engineering

over 2 years ago

Prompt Engineering, also known as In-Context Prompting, refers to methods for how...

The Transformer Family Version 2.0

over 2 years ago

Many new Transformer architecture improvements have been proposed since my last post...

Large Transformer Model Inference Optimization

over 2 years ago

Large transformer models are mainstream nowadays, creating SoTA results for a variety...

Some Math behind Neural Tangent Kernel

almost 3 years ago

Neural networks are well known to be over-parameterized and can often easily...

Generalized Visual Language Models

about 3 years ago

Processing images to generate text, such as image captioning and visual question-answering...

Learning with not Enough Data Part 3: Data Generation

over 3 years ago

Here comes the Part 3 on learning with not enough data (Previous...

FAQ

over 3 years ago

Learning with not Enough Data Part 2: Active Learning

over 3 years ago

This is part 2 of what to do when facing a limited...