Newsletter of Rishit Dagli - Issue #2

In this article, I’ll give you some opinionated tips to help you build a great open-source project you can start using. You can also use these tips for building hackathon projects.

A Transformer trained to produce all of the model parameters of a small ConvNet works really well for meta-learning! Allows you to decouple the complexity of the large task space from the complexity of individual tasks. Works very well for small target CNN architectures.

GANs are notoriously unstable to train. A widely used empirical heuristic to stabilize training is spectral normalization (SN). Zinan Lin explains why and how spectral normalization controls exploding and vanishing gradients and how you can further improve SN.

Realistic editing of real videos. Encoders are smooth at the local scale, generator tuning works great globally, and maintains alignment. Together they almost perfectly preserve the original video’s consistency. Maintains a higher degree of temporal consistency over the current state-of-the-art.

Models like GPT-3 in part are not aligned with their users. To make models safer, more helpful, and more aligned use an existing technique called reinforcement learning from human feedback (RLHF) to create InstructGPT obtaining more truthful and less toxic outputs/ Something really interesting: this has 100x fewer parameters than GPT-3!

A very interesting talk by Florian Marquardt as a part of his lecture series “Advanced Machine Learning for Physics, Science, and Artificial Scientific Discovery”.

Test Suites for Validating ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.

A Self-Supervised Speech Pre-training and Representation Learning Toolkit.