My transition journey into AI ML from a stagnant career in IT services
A framework and few secrets that worked for me to move in machine learning and artificial intelligence
PART 1 - The framework for transition
I had no fancy degrees, no great product brands, no $350K package, no internal leadership posting to apply for and no ‘aha’ moment — nothing.
I took actions to move from a legacy IT role (ERP middle management in 2014) into a startup founder in machine learning/AI. If I can most of you can as I had nothing fancy with me.
The framework I used can be used for transition into any mid-career digital career not necessary into Startup. The framework is below (Click the image to zoom):
Accept: I accepted that my role of a ERP Senior manager was stagnant. I could see no hikes happening. I felt under skilled in-front of young crowds. I heard stories of folks drawing 3x my salary and I was offered the work which was repetitive in which I was already an expert.
No one offered me more than 05-10% hike in the job market back then be it my onsite role in the UK or at India.
Explore: Back in 2012–13, you typically had a stack called as SMAC — Social, Mobile, Analytics and Cloud. I explored all of them at 101,102 levels to get skilled “independently” through online mediums and invested in a hi-spec laptop. I stopped exploring legacy skills and certifications in areas of ERP, BI, Project management, CRM etc.
Dig deep: I was a Senior Manager then and was off coding for nearly 10 years but Python made so much sense even starting new and I coded all the areas in python following codes of others in blogs, courses etc. Some codes made sense some did not but I did not stop, even made small hobby projects.
I understood that knowing things at “closed box” level will no more work in digital era. In long term I was reading basic paper summary for the technology I was skilling for like the paper on Resilient distributed dataset to skill on Spark.
Specialise: I kept analytics as my primary skill area and cloud (Azure) as secondary, I left actively pursuing other fields. I started creating my own blogs and articles as well. My blogs never became viral but it did serve as a basic proof of work and my own log of learning. I kept doing these without sight of immediate success or transition. I cleared Azure Data Scientist track as well.
Amplify: I mapped my personal goals with professional ones like where I wanted to live, what is the use of my life, how much money I wish to make and with what sacrifices etc.
This mapping and the continuous iteration of learning/skilling made me confident of taking up assignments in Universities initially and later in corporates mostly in training.
From training, I moved into small time consulting stuff like — “can you solve this real quick and in real less budget sort of stuff”. Iterated here again and moved up the value stream. From there went to an agency business model.
For me i wanted more control over my time and work so i went the entrepreneur way but this phase of “Map to execute” will also work if someone wants to change job or apply for internal jobs involving machine learning.
The advice I have is accept, explore, dig deep, specialize and amplify. This model may work in other areas as well not necessary machine learning.I have seen some mid career managers become excellent big data consultant for example.
In summary this is the distance i covered, maybe it is not the best outcome but definately these steps have provided me a recharge in my career and also in income and respect that earn in my daily work now -
Before moving in machine learning (Role/Work)-> Senior project manager managing legacy ERP but also managed a small consulting assisnment that was doing options analysis for Data lake. This was some where in 2014- 2015 in the UK.
Today my role and work is -> Run my own boutique data agency, most of my work involves machine learning consulting and training. (There were other factors that made me a “data entrepreneur”. But that’s for another blog on entrepreneurship.
Here is me in 2023–8 years since i started skilling holding two great books that helped me in machine learning skills
PART 2: (Each phase in details)
Now let us look each of the phases in depth and in more details. Click image to zoom.
“Accept” phase in details:
You would land in this phase if your career is stagnant. In my case the clues from the environment were very apparent. My pay remained constant, the only way I was making money was by changing country of jobs and chasing IT ERP projects for posting abroad from my base in India.
My short term and long term career was totally blur and I had no clue where my career would take be next year. I was not happy with what I was doing and felt everyday was repetitive for me.
This is where while traveling in a London Waterloo train I asked myself the below questions which I am now calling as —
Mid-career self-assessment: 10 sharp questions to confirm mid-career crisis in IT
Has my organization/project informed me that my role is becoming redundant?
How do I generally feel about my work?
Do I have anxiety about my personal finances, especially my future like retirement?
Am I happy with myself?
Do I feel underpaid and underworked?
Do buzzwords, especially in technology, irritate or scare me?
Do I always talk about the same things in a loop with my close colleagues during breaks?
When was the last time I enjoyed a holiday without thinking of work?
Am I as confident as I was 10 years ago?
Do I complain about my work at home/family?
Worked example from my situation back in 2014:
Below were my short “honest” answers for each of these questions as a sort of introspection in 2013. The key in this step is to be honest with yourself–
1. Has anyone formally informed me that my role is at risk?
My honest answer 1#
Yes, there were indeed communications about how automation and digital tech can change job roles especially my role(Senior project manager).
2. How do I generally feel about my work?
My honest answer 2#
Not very great, I felt I was at total control about what I was doing and seen as an expert in my role. But it had no punch-no excitement any more.
3. Do I have anxiety about my personal finances especially my future like retirement?
My honest answer 3#
Totally!! I had no clue how my earnings will beat inflation. I was very very worried about this point.
4. Am I happy with myself?
My honest answer 4#
No, I hardly laughed, and my smile was fake, I kept thinking always about X and Y. But no action.
I started the problem was my skill not my country of employment nor my employers.
5. Do I feel under paid and under worked?
My honest answer 5#
Hell yes, but I had left all hope of a higher payment slabs, thinking that’s how the market works with experienced folks.I was in total control of my work and did that very well but was bored as well.
6. Do buzz words especially in latest technology irritate or scare me?
My honest answer 6#
Yes, it did. I was from the era of closed source tech and modern open source tech buzz words irritated me as I was not in command over them.
One such example was Hadoop back in 2012. I realised i had a problem.
7. Do I always talk the same thing in loop with my close colleagues at break?
My honest answer 7#
Yes, I talked about employment visa and job opportunities in another country, I had changed 4 countries in similar roles (Germany, Singapore, UK, India) by then but still I kept talking on this.I started the problem was my skill not my country of emplyment nor my employers.
8. When was the last time I enjoyed a break?
My honest answer 8#
I enjoyed the Christmas slowdown working from home more. Vacations seemed like routine back trip to India.
9. Am I confident professioally as I was 15 years back?
My honest answer 9#
Yes and No — I was not sure. When I looked at my peers I felt I am confident but when I looked at young folks especially in deep tech and startups I felt less confident.
10. Do I complain about my work at home/family?
My honest answer 10#
A big yes, more like a kid who comes from school and dumps the day on family.
As you can see from my honest 10 questions framework and my honest answers then, I indeed needed to accept that I was at active risk not even professionally but from a mental health perspective as well.
I decided to do something about it.
I hope you would find this checklist useful in any other role as well.
Let me know did you find it useful and remember it is important to be honest while answering yourself. That is the first step to convince yourself for a change of status quo.
“Explore” phase in details:
In the explore phase, I focused on key digital frameworks available at that time (SMAC) and decided to get in both the technology and models using online learning.
Bear in mind, i am talking of 2014 when Youtube was not a rage and nor so much content was available in internet. Here like i said i explored both tech and models like below.
Technology i explored then in 2014 as part of my explore phase -
Basics of Python using anaconda python
Computer Vision using OpenCV
Azure compute basics as VM’s and used them for big data
Apache Spark and HDFS
Machine learning using Scikit Learn
Blockchain using Ethereum
Data Science basic mainly using Numpy, Pandas and Matplotlib
Models i explored in 2014:
Machine learning lifecycle
Product management in B2C
UX
Design thinking
CRISP DM
Distributed computing
Now, these were not deep dives but were exploratory approach trying to answer the basics of what, why and how aspects for each of the tech and model pillar.
For example in OpenCV i was able to understand what is computer vision and how it varies from image processing, how to draw bounding box on images and run a code about Haar cascades classifier. I never went deeper on any tech stack at this phase.
I did the same with models like in UX i explored how UX design matter, Why UX was then a rage among startups, How it differed from my IT world of UI, What are some basic principles for a mobile UX?
Hope you get it. The idea was to elicit interest, answer basic questions and also take a decision if i needed to skill deeper as part of specialisation.
I did another move in this phase, i dropped dead skills or mass-y skills which i thought was no longer in hot skill segment. Some skills i stopped upgrading myself were ERP, Any packaged software, I dropped my idea to get Program Management certification, dropped idea of moving to General Management up skill like MBA.
By the end of this phase which lasted 6-9 months i was convinced i needed to move into areas of machine learning and big data.
“Dig-deep” phase in details:
This is the deep skilling phase in which I did deep skilling choosing some skill from all the skills i explored in the previous phase of “explore”.
I broadly focused on two technical stack - machine learning and big data.
Big data - For deep skilling, I started reading the famous research papers or the core papers that underlined the technology. Like in big data i got deep skilled on Apache Spark and read in details the paper on Resilient distributed dataset or RDD’s.
I coded the basic example by looking at training codes and running them and observing the results. May python codes, gave syntax error and i learn the codes more by fixing the codes and “google-ing” the errors in sites as Stack overflow.
I did not do any personal hobby project as i was not looking to display my Git as part of portfolio but i will recommend my readers to at a bare minimum have a repo of codes that you like.
I also used a couple of books in my journey of big data - like Spark: The definitive guide and Designing Data-Intensive applications both from O’Reilly.
I also skilled myself on DP-203 which is Azure data engineering by using the guides available on the Azure site.
Machine Learning - I divided my machine learning journey into 5 large blocks viz - time series, regression, classification, clustering and neural network. I went deep inside this subject including the maths. In total i learnt about 25 models in machine learning and in deep learning my learning map was Perceptron model, ANN, Deep neural network, LSTM, RNN and reinforcement learning.
I relied heavily on scikit learn to code my my models and used lots of online trainings, webinars, youtube videos, courses from professor Andrew NG.
For deep learning the Youtube videos from Kris Naik and Coursera/Deeplearning.ai courses along with certification helped me a lot. I found that using Keras was easier than Tensor-flow but that was applicable then, today both these packages are interoperable.
My code editors were Jupyter notebook and VSCode. I like VSCode as it was useful for azure work as well as machine learning. For books i used mainly - Hands-on machine learning with SciKit learn and TensorFlow and Fundamentals of deep learning again both from O’Reilly publishers.
Artificial intelligence:
I had studied AI in my undergraduate engineering but it was then good old 15 years old. I mostly move to skill in AI after I was comfortable with supervised and unsupervised learning in machine learning. In the AI stack i was fully focussed over Azure AI services and consuming them over an app service. Other than that i did heavy readings in the areas of text mining, informed and uninformed search, RecSys - recommender systems mainly item based and user based collaborative filtering. (It is fun to know that text intelligence went to make impact later with large language models, in those days I was doing models as bag of words, word embedding and using NLTK’s and Keras based nets).
Here, i must admit that i was not coding from scratch but running and debugging codes from various sources of internet. (These were days before ChatGPT). There was another reason, that i focused myself more on various stages of machine learning rather than making myself good at Python programming.
“Specialise” phase in details:
In this phase, you focus on honing your expertise, diving deeper into a specific area within the vast digital realm. Having established a strong foundation in your chosen technology, you'll feel a pull to contribute to the community. This could manifest in various ways, from writing blogs to delivering talks or mentoring budding enthusiasts. The overarching aim? To solidify and validate your expertise.
As you journey forward, you actively carve out a space for yourself as a thought leader. Earning certifications or accolades in your domain becomes not just a goal but a testament to your dedication.
This stage is a game-changer in terms of role transition. Now is the time for you to elevate your visibility actively. Think about writing a deeply insightful article, reaching out for podcast appearances, or seizing opportunities to speak at seminars. It's also the moment to be audacious; start applying for new roles that align with your sharpened skill set, both within your current organization and outside. I'd recommend ramping up your networking efforts; make those essential phone calls, connect with industry stalwarts, and if the situation demands, be ready to switch jobs to align with your newfound path.
For me this phase was the phase, where i left my job to startup. But like I said at start, for you this may mean applying for an internal role or changing your job or even industry. In simple words “change kicks in here”
“Amplify” phase in details:
This part for me was to prioritise my life. What i wanted from my life in the next 15 years (from life and career) and what were the must have and good to have for me. My priorities pointed me at two things - viz I should startup and secondly, i should move back to India (I was in the UK on work visa during that time). For you it may be moving to a tech leadership role or anything that seems right for you for your career and life.
But i can get you a rough yard stick for this phase.
Prioritise your wellbeing and your family
See where are the opportunities for the tech skills you have identified in the “Dig deep” phase.
Make a list of things that are a must for your life. Do note some roles may impact your weekends and so can some organisations, are you ok? there is no right or wrong answer here.
Do not think about entry level role, back in 2016 many folks outside the machine learning domain thought data scientist as an exec role, it was an entry role in contrast. Be clear of the roles you are targeting and donot go with the designations.
Follow the trace of money. To be blunt, it does matter. See where the ROI fits your new skills.
Put a time-line and measure your risk. I was reckless, when i connect back. I would suggest you to balance out the risks a bit. One example is - check if there is an internal transfer in your current organisation for your new skills.
Be ready to loop to Accept phase every 3-5 years, use zig-zag skilling (non linear) growth etc in this phase. No skill is permanent skill till retirement in digital era.
So, this was in summary my transition story. The key take away is the model and then to map to your own career. Please do comment and tell me what do you think?
PS: Originally posted in Medium and connect me on linkedin if not done.
My Linkedin is here - Anirban Linkedin.
You can also consider sharing this post to someone you care for.