Discover more from The Overfit
Ensemble #1: Layoffs, ChatGPT, Research Picks, AI Summit
Why are layoffs happening? What is going on with ChatGPT? In this issue, I cover the most prominent topics in the tech and AI field in January 2023.
Ensemble is a monthly issue from The Overfit, aggregating the most relevant updates in the Data Science & AI industry for the given month, always with some personal takes on “why is this happening?” and “what will happen next?”
Last year, the tech industry had some notable layoffs. Many people thought the layoff waves would stop already, but January came hard with many big layoffs:
Why are layoffs happening? My personal view:
Tough past year: 2022 was the worst year for the U.S. stock market since the 2008 crisis. You can see in the image below a visualization of 2022 market changes. Except for the energy and healthcare sectors, every other industry took a big hit.
Uncertainty: Historically, high inflation + FED increasing interest rates result in recessions, which is why many economists believe one will happen in 2023. With that, companies and investors try to be more conservative, which makes getting funding more challenging (for growth projects — both in a startup and in larger companies), reducing growth expectations for the following year. On top of all that, we have political issues that we cannot predict when they will end, such as the Ukraine-Russia conflict.
Predictions: Given the tough past year, uncertainty, and less accessible funding, companies are compelled to make conservative estimations for 2023 by their shareholders. This requires companies to adjust their headcounts to maintain desired profit-to-cost / profit-to-staff ratios, avoiding a more significant impact in the case of an economic downturn, which finally results in layoffs, hiring freezes, or at least employee growth reduction.
Herd mentality: Last but not least, professor Jeffrey Pfeffer attributes the recent layoffs as a “social contagion” (source). As companies see others doing it, they feel compelled to follow. In my view, it combines this “mentality” with everything mentioned above, as the “uncertainty” for the next year has some logic and emotion behind it — behavioral economics in action.
Why Microsoft is considered odd: Microsoft is one company that surprised many with its layoff. The company is currently at an all-time profit high, and its revenue distribution is better than other big tech companies (such as Meta, with 97% of revenue coming from ads), with products such as Azure (31.3%), Office (23.7%), Windows (13.8%), etc.
Priority Hypothesis: In the layoff e-mail, Satya Nadella (Microsoft CEO) said they would “continue to hire in key strategic areas.” This indicates that they might shift priorities during 2023, shutting down some initiatives that will no longer require employees while focusing on others.
Overestimation Hypothesis: Let’s look at employee growth at Microsoft: 9.92% in 2019, 13.19% in 2020, 11.04% in 2021, and 22.1% in 2022 (note the spike). Whereas, in gross profit growth, Microsoft had: 16.89% in 2019, 19.52% in 2020, and 17.06% in 2021. This indicates that they expected more growth during 2022 and decided to make more conservative estimates for the next year, given the market uncertainty and potential business priority changes, as mentioned before.
(even though there might be a priority change, the overestimation hypothesis is less farfetched for me, given the solidity of Microsoft’s business — closing one small project or another would not make such a significant difference in HC)
Despite all this, out of the laid-off people, "72 percent have found new jobs within three months. Even more surprising, a little over half of them have landed roles that pay more than what they were earning in the jobs they lost" (source)
Will this continue? I expect layoffs to continue, especially during February (as it matches the budgeting cycle for some companies). We will likely only recover from this scenario after the market uncertainty declines significantly and companies can get more confident in their growth.
What is it? ChatGPT is a large language model that is trained to generate human-like text. It can be used for a variety of natural language processing tasks such as language translation, text summarization, and question answering. It is trained using a variant of the GPT (Generative Pre-training Transformer) architecture and is fine-tuned for specific tasks using additional training data.
(ChatGPT generated the above description)
This technology, released by OpenAI, caught a lot of attention — one million users within a week of its launch — because it provides excellent responses to many prompts, and it made the public pressure Google to show advances to their search engine in similar ways.
Still, there are many limitations to ChatGPT. For instance, it will agree when given a misleading prompt:
In the example below, I asked it to generate code to monitor concept drift, and it generated some code that “makes sense”; However, i) it does not follow “best practices” for detecting concept drift; ii) some critical parts are left out, such as the “handle_drift” function; iii) It is not functional, as the threshold variable was never declared (is it global?).
Still, with more guidance and some tweaks, it can generate useful code (at least to serve as a quick start), especially for more straightforward, well-defined tasks.
Finally, OpenAI announced the transition from 100% free to a freemium business model, in which you pay $42 a month for higher availability, response speed, and access to new features.
I will continue to use ChapGPT on some applications (the free version, most likely), and I am excited about the subsequent releases from OpenAI and possible competitors. This is a promising year for AI-based applications, especially in Natural Language Processing and Computer Vision.
In a future post, I will go through the details of how ChatGPT works, its predecessors, and expectations for the next versions. Click below if you want to read that and did not subscribe yet! :)
CICERO: “An agent that can play at the level of humans in a game as strategically complex as Diplomacy is a true breakthrough for cooperative AI.” (Meta AI)
Teaching Algorithmic Reasoning via In-context Learning (arxiv) [actually from last year]
Deciphering Clinical Abbreviations with Privacy Protecting ML (Google AI)
There is an exciting event on January 26th (tomorrow if you received when I sent this issue) with the following topics:
Data-centric AI (a chat with Andrew Ng)
Operationalizing Machine Learning at a Large Financial Institution
Large Language Models (LLMs) to Production
Get Your Company Started with AI: The Right Way
That’s our wrap-up for January 2023!