Akira's Machine Learning News

By Akira's Machine Learning News -- by Akihiro FUJII : Manufacturing Engineer / Machine Learning Engineer/ Master of Science in Physics / ExaWizards Inc.

I introduce articles and papers related to machine learning every week. I also publish monthly and semi-annual summaries of the most influential research in that period.

I introduce articles and papers related to machine learning every week. I also publish monthly and semi-annual summaries of the most influential research in that period.

By subscribing, you agree with Revue’s Terms of Service and Privacy Policy and understand that Akira's Machine Learning News will receive your email address.

38

issues

#38・

Akira's Machine Learning News - Issue #38

Featured Paper/News in This Week.A 3D-Transformer that can be directly applied to molecular structures has been proposed. The attention weights can be adjusted according to the interatomic distances, and the computational complexity does not seem to be that h…

 
#37・

Akira's Machine Learning News - Issue #37

Featured Paper/News in This Week.The Self-Attention part of the Vision Transformer is interpreted as the "token mixing part", and it seems to perform reasonably well even when pooling is used as the simplest token mixing method. Personally, I feel that the mo…

 
#36・

Akira's Machine Learning News - Issue #36

Featured Paper/News in This Week.SimMIM, a model for image pre-training with a structure similar to a masked language model, has been presented. The concept is similar to MAE introduced last week, but this is a simpler implementation. It may become more commo…

 
#35・

Akira's Machine Learning News - Issue #35

Featured Paper/News in This Week.A method is proposed to mask the image and pre-train the model to recover it, like BERT. 75% of the image is masked and only 25% of the unmasked image is input to the encoder, which seems to be memory friendly.An image generat…

 
#34・

Akira's Machine Learning News - Issue #34

Featured Paper/News in This Week.There have been a study in the past that have shown that ViT classifies with a more human-like behavior than CNN, but now a new study has been published that shows that ViT correctly classifies even when perturbed on a patch-b…

 
#33・

Akira's Machine Learning News - Issue #33

Featured Paper/News in This Week.A new dataset for self-supervised learning has been released that can be used for commercial purposes and is portrait rights friendly. As a member of the industry, I am very grateful for such a dataset, as large-scale data suc…

 
#32・

Akira's Machine Learning News - Issue #32

Featured Paper/News in This Week.Methods to improve the performance of zero-shot inference have been presented. Since GPT-3 zero-shot inference is used in many applications, any improvement in the performance of zero-shot inference may have a significant soci…

 
#31・

Akira's Machine Learning news - #issue 31

Featured Paper/News in This Week.A published study shows a sudden improvement in generalization performance from random results: overfitting starts at about 10^2 steps, and a sudden improvement in generalization performance from random prediction is reported …

 
#30・

Akira's Machine Learning news - #issue 30

Featured Paper/News in This Week.A study has been presented that uses a few sketches to adjust the parameters of a GAN. Then, it will learn to generate images to match the sketch images.Research has been presented on the differences in features acquired by Vi…

 
#29・

Akira's Machine Learning news - #issue 29

Featured Paper/News in This Week.A series of articles on MLOps with Pytorch Lightning is available. It covers many things, from models using W&B to CI/CD using GitHub actions.Google Research has published a paper describing the methodology and details of …

 
#28・

Akira's Machine Learning news - #28

Featured Paper/News in This Week.A hardware (camera) specific adversarial attack method has been presented. It seems to generate adversarial noise by using a surrogate model that reproduces the embedded hardware and taking differential values. As with physics…

 
#27・

Akira's Machine Learning news - #27

Featured Paper/News in This Week.A paper reporting the use of Transformer in industry has been published. They used Unified Visual Embedding, which embed the features into a common embedding space, and trained it on a total of 1.3 billion different pieces of …

 
#26・

Akira's Machine Learning news - #26

Featured Paper/News in This Week.A study of unsupervised learning on a large amount of video data is presented, using Transformer to train well-designed tasks in both temporal and spatial directions. While handling 6 million pieces of data, the improvement in…

 
#25・

Akira's Machine Learning news - #25

Featured Paper/News in This Week.There is a research on training transformers with masked language models such as BERT on image data.Personally, I think this is a good application of Vision Transformer, since it treats images as tokens and processes them in a…

 
#24・

Akira's Machine Learning news - #24

Featured Paper/News in This Week.There is an article that says that the AI tools created for COVID-19 was completely useless. The main cause seems to be leakage and data quality, but it reminds me that it is difficult to create something that works well in a …

 
#23・

Akira's Machine Learning news - #23

Featured Paper/News in This Week.A paper on MLOps anti-patterns has been published. Records of failures are very valuable, but they are not often published. This is a very good paper.A method for tracking using all the object candidate outputs of object detec…

 
#22・

Akira's Machine Learning news - #22

Featured Paper/News in This Week.Strategies have been announced that allow scaling even in large models. Since NAS is not possible for large models, it may become important now that huge models have become a trend.Some research has been done on applying contr…

 
#21・

Akira's Machine Learning news - #21

Featured Paper/News in This Week.In pre-training, the Winning ticket of the lottery hypothesis seems to exist regardless of whether it is supervised or unsupervised. Since pre-training models are usually very huge, it may be possible to use a winning ticket (…

 
#20・

Akira's Machine Learning news - #20

Featured Paper/News in This Week.A paper has been published showing that capsule networks are not as robust as CNNs. The core technology, Dynamic Routing, seems to have a negative impact on accuracy.In the lottery hypothesis, where a small number of useful in…

 
#19・

Akira's Machine Learning news - #19

Featured Paper/News in This Week.There is a paper out that suggests that the machine learning community is only focusing on improving models and neglecting the impact that bad data can have. In my experience as a practitioner, improving the data has a bigger …