Through my own job, my network, and my other activities, I’m exposed to the industry a lot. I have the chance to talk to other insiders, enterprise/bank IT leaders, and many many more. I also consult as a side hustle and build one or the other website from time to time. I don’t do this full-time, yet, but it may give me some credibility.
This section is here to showcase scenarios, skills, and business ideas that I view as opportunities for someone who wants to make a change and go the entrepreneur route or simply upskill themselves.
Upskill Yourself - Data Engieering
Data Science and Machine Learning are on the rise but they depend on data. The issue with data is that a lot of companies collect huge amounts of it but then store it in silos without making it really usable. This is where data engineers come in. They collect data (extract), transform it, and then load it into another data store (usually pretty centralized ones, also called “data lake”) where data scientists or ML engineers can fetch it from and use it to build their models. Data engineers are also responsible to design the schema of how the data is stored and create streaming pipelines to transfer it to where it is actually needed.
The demand for data engineers has far outgrown the one for data scientists and ML engineers, and there is a huge shortage (as for all software engineering roles). What makes the situation worse is that there are estimations that two to three data engineers are needed for each and every data scientist or ML engineer employed. Companies slowly but steadily adapted to this shortage and have increased salaries over and over again to attract new talent. The Robert Half 2020 Salary Guide, for example, lists data engineers with a base salary recommendation of $120.000 - $222.000 while the Dice Salary Report 2020 lists them with an average salary of $113.249.
Data Engineers usually work with Python, Java, or Scala, with Python being the most prominent language choice. Companies with a modern data engineering stack will usually require knowledge of Airflow, Spark, RDBMS, and NoSQL. Really modern tech stacks will replace Spark for Kafka or Snowflake. All positions will usually require some knowledge of containers, databases & data warehouse optimization, data modeling, and the actual business domain.
If you are looking for a role change, the data engineer might be a great choice with a lot of potential for the future. Here are some sources for further reading: Quanthub
Self-employment - The Serverless JAMstack Consultant
Taking a look at the framework trends on BuiltWith
reveals that the majority of public websites are still built with PHP. From my personal experience, I can add that in the enterprise sector, a large portion of projects is still built with older technologies that don’t even use SPAs but settle on server-side template rendering. A smaller portion leverages SPAs that are served by a Java application server (e.g.) with a “backend for frontend” attached. In the agency space (according to some of my connections) PHP is dominant, followed by more modern approaches leveraging Python (Django) or Node and SPAs. Current state-of-the-art approaches include deploying everything within containers and running them on huge Kubernetes clusters. Frontends are still delivered by application servers or are more modernly served by an httpd or nginx that’s also running within a container. The frontends fetch data from multiple services running within the cluster but developer experience-wise not much has changed.
Although not applicable to each and every problem (currently), the JAMstack brings a huge opportunity to both businesses and developers. It aims to offer scalability, safety, a better developer experience, and cost-saving. If we take a closer look at it, it is absolutely able to deliver on all of it. The most important thing the JAMstack enforces is a strict separation of backend and frontend by leveraging the API economy and focussing on the end-user experience in the project at hand. You can read more about it in an article I recently wrote
After this lengthy introduction, let me get to the point. I think that there is a huge opportunity in consulting companies on the JAMstack. Leveraging modern static site generators and serverless functions as a service (FaaS) can bring a lot of value to frontend teams and help them to transform into teams that can safely work on their own, with a reduced need for skills (no more traditional backend skills needed), faster results, and a better user experience (website load speeds and enhanced SEO). If you sell yourself as a consultant and architect who can help companies transform their architecture into a cost-saving system while reaping the benefits the JAMstack offers and those of the cloud, you can charge really high amounts of money, simply based on the immense value you provide.
Last but not least, here are a few skills I think you need in order to be successful in pursuing this career:
Knowledge of AWS Lambda/Amplify
If you want to work together with larger companies, you won’t get around AWS. They currently simply offer the best serverless and JAMstack experience of all the large cloud providers. Firebase is cool, but you’ll find more companies already on AWS than there are on Google Cloud.
Knowledge of Netlify / Vercel
This one is optional, but if you plan to also work together with small to medium-sized companies, Netlify or Vercel offer a better experience for smaller teams and smaller budgets. They are based on AWS but provide an easier-to-use interface and a better developer experience than pure AWS services.
Some DevOps skills
You’ll want to be able to offer the full-stack experience, from project set up to the deployment of the finished product. Serverless offers already make this process pretty easy, but you want to be able to advise the teams you work with on how they can ensure that their projects are always built, tested, and delivered correctly.
Good HTML and CSS foo (but this is actually optional)
Frontend work involves HTML and CSS. If you plan on working together with the teams you consult, you should consider serving the full frontend stack. Being able to build good-looking prototypes and maybe even working on the project yourself is another good selling point for your overall service.
Skills in Next.js/Gatsby/Nuxt and/or Hugo
A good portion of system design capability
You want to sell an architectural pattern that helps to transform the way an organization builds software. You should be able to design an architecture that fits the project at hand. It won’t be as complex as a full system architecture for something deployed to Kubernetes but a few Lambda functions, external APIs & databases, existing microservices, API Gateways, and such will still be a part of the system you create. You will need to be able to design all that.
One core concept of the JAMstack is to put everything under version control that you possibly can. Git is an essential part of this concept, and thus it makes a lot of sense to know at least the very basics of it.