According to Harvard Business Review, Data Scientist is the sexiest job of the 21st century. With exponential growth in the amount of data generated every day, the world needs specialists who can extract value from that data.
Data science had a tremendous impact on many industries, but machine learning has always been a key driver to digital transformation and automatization.
A career in machine learning might seem appealing, especially if you look at some facts:
- According to LinkedIn, artificial intelligence specialists took first place among the 15 fastest-growing jobs in the United States for 2020;
- Machine learning Engineer was on top of Indeed’s list for the best jobs in the US in 2019;
- According to Indeed, Machine Learning Engineer job openings grew 344% between 2015 to 2018, and have an average base salary of $146,085.
Machine learning is now everywhere around us: in music, healthcare, social networks, even in chess. The number of applications is huge, and it keeps getting bigger as more and more industries adopt this technology to tackle their problems. The demand for machine learning specialists is constantly growing.
If you’re not convinced about how widespread machine learning is, you can check out these examples of using artificial intelligence at some of the top companies to improve their operations, services and products. You will be pleasantly surprised.
Being part of a rapidly growing artificial intelligence environment is an attractive career path, but how can you enter that path?
You might have already heard that machine learning engineers are expected to be good at math and statistics, know programming languages, have a bit of business sense and solid research skills. It’s all true, but don’t let it overwhelm you.
You don’t have to be proficient in all of the above to start your career as a machine learning engineer. It takes time, effort, and practice to meet all of the qualifications.
Everyone has a different ML learning path
Each of us has a unique background, knowledge and experience. There’s no universal recipe for everyone to learn machine learning. My personal experience taught me that there are two distinctive types of learners:
- Those who have a chance (and resources, especially time and money) to fully dedicate themselves to learning. For such learners, this typically means getting a degree and spending multiple years studying;
- Those who lack the resources to get a degree. They might be individuals who already have jobs, families and, understandably, can’t pursue a full-time degree. This group also includes those who simply stay away from getting a degree because they can’t cover the costs of education.
If you can afford a degree (financially and time-wise) and you’re excited about machine learning, then just go ahead and do it. You will enjoy new knowledge, practice your new skills, and enter the field smoothly. Obviously, this is the most convenient scenario for your future career and development.
What if you can’t afford a degree? You can’t become a machine learning engineer? I don’t think so, because that was my situation, too. My own experience is proof that it’s possible. I also have a friend who worked for a long time as a barista, but then decided to become a machine learning engineer. And he did it! After 11 months of part-time studying, without quitting his full-time job, he got his first ML job offer and joined a tech company in a new desired role.
As you can see, no matter what your particular case is, you can get a machine learning education and break into the field if you’re dedicated. In this article, I’ll give you some tried-and-tested guidelines to make this transition easier.
The first group of learners usually has a well designed curriculum to follow, and extensive support from academic staff, but we won’t spend much time talking about it. We will only slightly touch various degree options that are worth considering.
Apart from that, we’ll mostly focus on the second group of learners and their needs. Such students usually follow a self-designed learning path that mostly consists of MOOCs, books, and bootcamps. In this article, we’ll guide you through all options you can learn from. We’ll consider their pros and cons, and help you pick the format that best fits your learning style and requirements.
Not surprisingly, the very first question that most self-taught students face is about the design of their learning course. On a high level, it usually means the topics and subjects in the studying plan. A carefully planned syllabus is important, and we’ll talk about the subjects that every machine learning engineer must learn and know.
Besides an intuitive subjects list, there’s another, no less important concept that each and every self-taught student should understand at the very beginning of their journey – you should learn using the top-down approach, and not bottom-up. Here’s why:
Top-down approach to Machine Learning education
The top-down approach is about moving from a high level solution to the nitty-gritty of an applied method. It’s the direct opposite of the traditional academic way of learning, where you learn theoretical principles first, and only then do practice.
The top-down approach, in contrast, will not require investing a significant amount of time into studying theory before moving on to practical aspects. Instead, it’s about obtaining proper tools (usually libraries of your favourite programming language) and working on getting things done before even learning how everything works under the hood. Your end goal is to solve a tangible task using the high-level tools and methods available to you.
This approach has been working great for many learners who decide against a traditional college degree. You first bring to light what applied machine learning is, get excited about the opportunities in this field, get familiar with programming principles and the main libraries used. Once you go through this process, and become more confident and motivated, only then you dive into the theory side and learn the mathematical concepts behind the algorithms.
A great example of top-down in action is the The Machine Learning Crash Course created by Google. It gives you a hands-on approach to learning. Using real-world case scenarios, visualization and a ton of exercises, learners can enjoy the learning process without getting overwhelmed by complex mathematical concepts. It’s fast, it’s fun, it’s efficient and, most importantly, it’s a great way for self-taught learners to get familiar with machine learning.
Massive Open Online Courses (MOOCs)
The machine learning crash course from Google is an example of a MOOC (massive open online course). There are plenty of MOOCs available online. Some are designed by top universities (check out this page to browse among the MOOCs from Stanford), some are released by individual contributors or even independent schools.
The best resource to find courses from individual contributors is Udemy. It has the world’s largest selection of courses, and around 130,000 online video courses with new additions published every month. The machine learning section has a vast number of course options to pick from. The prices are very affordable. Each course has a review and a rating score that help you make the right decision and eliminate low-quality content.
As for schools and universities, there are many options to consider. MOOCs from schools and universities are usually more expensive. It’s completely understandable: we pay for the highest quality of learning materials created by experienced academics.
The syllabus might stay the same, but with a MOOC from a school or university you usually have an opportunity to get feedback on your work. It usually means a review on projects, or a chance to address your question to an expert in the field. Some places even offer office hours and live streams with staff, which can be quite helpful.
One of my favourite sites to learn machine learning is Udacity. Their so-called nano-degrees are 3 to 6 month courses, with plenty of practice and a good, concise portion of theory to back up practical skills. Praktikum is another great place to study machine learning that I personally like.
Udacity and Praktikum are just a few examples of data science schools. Udemy is not the only source of MOOCs created by individual contributors. Stanford is not the only university that provides online courses. There are many other schools, platforms and universities open to us – almost too many, which might cause decision paralysis or FOMO.
Because there are so many good resources, the process of finding the right place to study is challenging and time-consuming. Hopefully, now you have a good sense of how MOOC providers differ and what to look for. Although MOOC is a great format of learning, it’s not the only option.
Another way to learn machine learning is via bootcamps. Participating in a bootcamp is similar to taking place in a marathon. It’s longer than a regular sport activity, much more intense, and very immersive. But be careful before you pay for one, because bootcamps are not for everyone.
In terms of involvement, the experience you get in a bootcamp will be similar to getting a university degree, since bootcamps also require full dedication to the learning process.
The main difference between a degree and a bootcamp is the time it takes to finish them. The whole bootcamp process usually lasts no more than 6 months, with an average of 3-4 months. Some bootcamps even guarantee that their students will get a job offer by the end.
How can a bootcamp lead to the same outcome (a job offer) as graduating from a university? Even though you spent less time studying (and paid less for it), the end result is the same. Isn’t it suspicious?
Well, the secret here is that most bootcamps use the top-down approach to education. Students gain practical skills in a short period of time, allowing them to get a job offer after the course, and then keep expanding their knowledge while they’re working.
In terms of pricing, bootcamps are usually a bit more expensive than regular MOOCs. From my experience, I can say if you have a chance to jump into the intensive environment of a bootcamp, you will definitely benefit a lot from it.
Springboard is one of the bootcamps that offers a machine learning program. The course takes 6 months, and requires up to 15-20 hours a week to complete. If the approach that bootcamps propose matches your way of studying, I would recommend checking out this list of bootcamps to browse available programs.
Bottom-up approach to Machine Learning education
Alternatively, if you have the time to invest into studying, the necessary financial resources, the patience and desire to learn everything from the very basics, you should consider the bottom-up approach. It will help you build a firm foundation that will ensure a smooth immersion into the ML workforce.
Bottom-up is a traditional approach to learning. First you learn the theoretical part, and only then slowly move to the practical side. In terms of machine learning, it means:
- getting a solid mathematical background (including calculus, discrete math, linear algebra, probability theory and statistics);
- gaining proficiency in programming (usually, in Python);
- learning the core principles of backend / software development;
- understanding databases, data querying and processing;
- mastering the principles of data scraping, cleaning, manipulation and exploration;
Only then you might move to some kind of “introduction to machine learning” course that will teach you the basics for machine learning algorithms.
Does it take time? Yes, and a lot of it. It will take at least 3 years.
But do you need a degree to get a job offer, and start a career in machine learning? No, you can land a job offer without a college degree.
So what are the benefits of bottom-up ML education? Will you get profound knowledge? Sharpen the skills needed to accelerate your future career? Absolutely, yes! Your knowledge and skills will be well-rounded and complete. This will be a boost for your career.
Will the basics that you learn now help you understand future advances in ML? Yeah! The machine learning field is rapidly changing. The solid foundation gained from a college education will help you stay on the edge of progress.
It might be very hard to self-design a full-time syllabus which will cover all of the aspects needed to break into the machine learning field. With no proper background and no support from experienced staff, it might be impossible for some. That’s why most learners who choose a bottom-up approach to learning, stick to a traditional academic path and get a degree.
Online Master’s Degree
I won’t spend much time talking about university degrees with conventional classroom classes. Most of you have a good sense of what it is and how it works. Instead, I’d like to talk about online education. There’s been a big rise in this stream recently, with a lot of new opportunities for students. Here are a few benefits of online education:
- It blurs the borders. As long as you have internet access, you can get the world’s best educational programs from top universities;
- It’s cost efficient. The price of getting a degree is much more affordable compared to the traditional, full-time, on-location format;
- All decisions are up to you. Students have flexibility in their schedule. You decide when to study, how long, and where;
- It’s convenient. No need to spend time commuting and doing small-talk, you can focus on learning. Most work materials are in a digital format (e-books, videos, quizzes, practice tests), so you can always recap and refresh.
The advantages of online education are obvious. We all want to pay less, know more, be efficient and have more time to ourselves. Online education gives us these opportunities.
It’s important to say that switching to online classes doesn’t affect the program quality, and doesn’t lead to limited communication and support from staff. In contrast, online students report higher rates of satisfaction in regards to connection with classmates and academic staff. Full-time students who moved to online format did not see any changes in terms of the educational program.
I personally love the new online format to getting a degree. Here are a few recommendations where you can look for an online degree for you:
Many other universities and colleges offer online degrees. Take your time to browse through available programs and select those that meet your requirements. This ranking might be a good starting point for you.
Machine Learning concepts and topics to learn
You now should have a good understanding HOW you can learn machine learning. But WHAT exactly should you focus on? That’s what we’re going to discuss in this chapter.
As you might expect, focus for students who follow the top-down approach differs from those who approach machine learning bottom-up. Let’s look at what a syllabus for both types of students might look like.
ML curriculum for the bottom-up approach
The conventional syllabus for those who take bottom-up approach might look like this:
- Linear Algebra
- Discrete mathematics
- Probability theory
- Descriptive and Inferential Statistics
- Basic Python
- Advanced Python (OOP, main libraries like Pandas, Numpy, Matplotlib, etc…)
- Second programming language: C++ or R (optional)
- Machine Learning:
- Supervised algorithms (regression, classification)
- Unsupervised and semi-supervised algorithms (clustering, dimensionality reduction, graph-based algorithms)
- Deep learning (CNNs and RNNs)
- Reinforcement learning (dynamic programming, Monte Carlo methods, heuristic methods)
Topics in the curriculum may vary, but the above program structure is quite common and well established.
ML curriculum for the top-down approach
Top-down practitioners should carefully consider their syllabus. Their time is precious, resources limited. It’s crucially important to target only those parts of the program that will maximize the end result.
Depending on the stream you’d like to end up in, your learning plan will be different. Always remember what your end goal is: it’s about getting the job done in a fast and efficient way. It will always help you stay focused on the end result and tailor your learning path accordingly.
For top-down practitioners, I would suggest the following learning plan:
- Define stream(s) in machine learning you’re interested in;
- Understand the structure of the problems in a given stream;
- Look for best practices and try to understand them on a high level;
- Find a problem that you can solve applying these best practices. Use high level tools when solving.
For example, you love crunching numbers and want to apply machine learning to this. In this case, classical machine learning is your stream of choice.
As a first step, you do brief research and figure out the methods / algorithms with best performance in your problem domain (domain of numbers), and apply this algorithm on a high level to a problem that you have.
In case you don’t have a problem to work with, find it at one the largest machine learning communities – kaggle. It has a list of competitions with a list of problems that you can solve. When the problem is defined and clear, look for the Python libraries from where you can import the best performing algorithms and apply them to your problem.
Another example. You love working with images and videos, so you’d like to learn machine learning and link it to your digital passion. Given that, computer vision engineering is your ML stream of choice. Google the problems that computer vision engineers solve. Understand on a high level what these problems are, and which algorithms can tackle them. Find these algorithms on github (there are many open-source solutions out there) and apply it to a problem from kaggle.
You will face a lot of pitfalls on your way. Don’t feel frustrated because of that. Instead, I’d suggest doing the following:
- Understand what stops you from solving a problem;
- Try to find a quick solution;
- If there are none, find a proper course that will help you get better prepared for the problem you faced.
Let’s continue our second example with images and videos. After research, you find yourself working in computer vision. You understand that computer vision engineers can solve multiple problems: they can classify images, detect objects in images, and also semantically segment objects from images.
You feel like the classification problem is not as complex as others, so you’d like to start with it. You have no problem to work with, so you move to kaggle.com for inspiration. After some more research, you note an interesting project: creating an image classifier to distinguish dogs from cats.
You looked for the best practices for image classification, and it leads you to Convolutional Neural Nets (CNNs). You start looking at what CNNs are, and find that they’re quite complex. Knowing that CNNs are the thing that you need, you find a free online course that explains what CNNs are about. Later on, armed with the knowledge about CNNs, you now have everything to get back and create a desired classifier.
I hope that you now have a feeling of how your workflow should be designed.
There are many sources of information available to us on the internet. If you browse through the web, most guides that are aimed to prepare you for a machine learning career will be either solely about the bottom-up approach, or just simply an advertisement of a particular school where you can study. Neither source of information should be taken seriously.
In this article I highlighted that there are two distinguishable ways you can use to approach machine learning:
- top-to-bottom (problem -> practice -> theory),
- bottom-up (theory -> practice -> problem).
My recommendations are based on years of experience and teamwork in the machine learning field. I hope that you will benefit from my advice, and select an approach that best fits your needs.
Best of luck!
ML Experiment Tracking: What It Is, Why It Matters, and How to Implement It
10 mins read | Jakub Czakon | Posted November 26, 2020
Let me share a story that I’ve heard too many times.
”… We were developing an ML model with my team, we ran a lot of experiments and got promising results…
…unfortunately, we couldn’t tell exactly what performed best because we forgot to save some model parameters and dataset versions…
…after a few weeks, we weren’t even sure what we have actually tried and we needed to re-run pretty much everything”
– unfortunate ML researcher.
And the truth is, when you develop ML models you will run a lot of experiments.
Those experiments may:
- use different models and model hyperparameters
- use different training or evaluation data,
- run different code (including this small change that you wanted to test quickly)
- run the same code in a different environment (not knowing which PyTorch or Tensorflow version was installed)
And as a result, they can produce completely different evaluation metrics.
Keeping track of all that information can very quickly become really hard. Especially if you want to organize and compare those experiments and feel confident that you know which setup produced the best result.
This is where ML experiment tracking comes in.Continue reading ->