View profile

AI Scholar Weekly - Issue #49

AI Scholar Weekly - Issue #49
By Educate AI  • Issue #49 • View online
New Model Achieves Comparable or Even Better Performance than CNNs; The State of AI Ethics 2021 Report; Learn if Facial Recognition Systems Used Your Photos; A Promising and Practical Technique for Training ML Models on Low-Powered Mobile Devices; How to Integrate ML to Cloud Resources (and more).

Top AI Research Papers This Week [Image credit: Pixabay] [Image credit: Pixabay]
#1 This New Model Achieves Comparable or Even Better Performance than CNNs for Vision Transformers
Transformers are largely successful for language tasks. But, they have also been explored for vision tasks through Vision Transformer (ViT), the first fully-transformer model that can be directly applied for image classification. However, ViT achieves inferior performance compared with CNN’s when trained from scratch on a midsize dataset.
To address the challenge, researchers have proposed a new T2T-ViT model that can be trained from scratch on ImageNet and achieve comparable or even better performance than CNNs.
T2T-ViT effectively models the structure information of images and enhances feature richness, which overcomes the limitations of ViT. It introduces the novel tokens-to-token (T2T) process to progressively tokenize images to tokens and structurally aggregate tokens.
The new approach achieves superior performance than ResNets and comparable performance with MobileNetV1 with a similar model size when trained from scratch on ImageNet and paves the way for further developing transformer-based models for vision tasks.
#2 A Promising and Practical Technique for Training ML Models on Low-Powered Mobile Devices
If you want to implement split learning for model training in mobile devices, this recent paper proposes SplitEasy, a new framework designed to assist you to do that!
In fact, this is the first work that discusses the implementation of split learning techniques and reports measurements of the associated computational and communication costs on mobile devices. Split learning has recently emerged as a promising technique for training complex deep learning (DL) models on low-powered mobile devices.
Researchers specifically highlight the theoretical and technical challenges that need to be resolved to develop a functional framework that trains ML models in mobile devices without transferring raw data to a server in the paper. They also demonstrate how SplitEasy can train models that cannot be trained solely by a mobile device while incurring nearly constant time per data sample. As a developer, you can run various DL models under split learning setting by making minimal modifications with SplitEasy. Read the paper: A Practical Approach for Training ML models on Mobile Devices
#3 A Machine Learning Data Processing Framework
Scholars with Google, Microsoft, and ETH Zurich have presented, a framework for building and executing efficient input data processing pipelines for machine learning jobs at scale.’s programming model enables users to build diverse input pipelines by composing and customizing operators. executes input pipelines as dataflow graphs and applies static optimizations that improve end-to-end training time for state-of-the-art models.
At Google, researchers have been using in training research and production ML models since 2017. Currently, the system implementation consists of over 15K lines of Python and over 40k lines of C++ (excluding test code).
The framework is used for data processing by the majority of TensorFlow training jobs in Google’s fleet. These jobs run in production clusters, spanning a variety of application domains including image classification, translation, and video content recommendation, and using various types of ML training algorithms.
The paper shows that features, such as parallelism, caching, static optimizations, and non-deterministic execution are essential for high performance. Read the paper: A Machine Learning Data Processing Framework
 #4 Natural Language Understanding at Scale with Spark NLP
This paper introduces Spark NLP, a Natural Language Processing (NLP) library built on top of Apache Spark ML. Spark NLP provides simple, performant, and accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment.
The library has been downloaded more than 2.7 million times and is experiencing exponential growth since January 2020. Currently, it’s used by 54% of healthcare organizations as the world’s most widely used NLP library in the enterprise.
Spark NLP comes with 1100 pre-trained pipelines and models in more than 192 languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster.
Spark NLP’s annotators utilize rule-based algorithms, machine learning, and deep learning models which are implemented using TensorFlow that has been heavily optimized for accuracy, speed, scalability, and memory utilization.
Spark NLP library has two versions: Open source and enterprise. The open-source version has all the features and components that could be expected from any NLP library, using the latest DL frameworks and research trends. Enterprise library is licensed (free for academic purposes) and designed towards solving real-world problems in the healthcare domain and extends the open-source version. Read the paper: Natural Language Understanding at Scale with Spark NLP
Other Great AI Papers
Digitized healthcare data have demonstrated potential success in enhancing health decision making and healthcare delivery. However, health data are usually noisy, complex, and have heterogeneous forms, yielding a wide range of healthcare modeling tasks. To solve the challenge, new research now presents PyHealth, an open-source Python toolbox for developing various predictive models on healthcare data with fewer than ten lines of code.
This survey is the first that focuses on personality-aware recommendation systems. Read  full paper
Since their introduction, GANs quickly became the gold standard in implicit generative modeling. But, what makes GANs work is still elusive. This research presents an alternative perspective on the training of generative adversarial networks GANs. 
How to integrate Machine Learning (ML) to cloud resources management is an important field today. This overview explores the most important cloud resources management issues that have been combined with ML and present many promising results.
A new study providing readers with a set of new perspectives to understand deep learning, and intuitive tools and insights on how to use adversarial robustness to improve it. In-depth Review of Adversarial Robustness in DL. 
AI Resources
The Definitive Resource for all Things ML in the Galaxy. The Engineers Guide to Machine Learning
The State of AI Ethics Jan 2021. Download Report 
A Python package for functional data. The package is publicly available on the Python Package Index and Github.
Essential Algorithms Every ML Engineer Needs to Know. Read here
Top AI News
The new type of neural network could aid decision making in autonomous driving and medical diagnosis. Read more
Here’s a way to learn if facial recognition systems used your photos. Read more
California to license driverless cars operated by Artificial Intelligence. Read story
Amazon Plans to Use a Unique AutoML Approach to Makes AI Accessible to Enterprises. Read more
About AI Scholar Weekly
Thanks for reading. Create a ripple effect by sharing this AI Scholar Newsletter with someone else, and they will also be lit up!
If you have suggestions, comments, or other thoughts, we would love to hear from you, email me at, tweet at @cdossman, like us on Facebook, or connect with me on LinkedIn
Did you enjoy this issue?
Educate AI

AI Scholar Weekly brings you everything new and exciting in the world of Artificial Intelligence, Machine Learning, and Deep Learning every week for free.

In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue