An introduction to machine learning

The words “machine learning” might conjure images of hyper-intelligent robots in the distant future, but the technology is a reality now. Algorithms powered by machine learning and drawing on big data enable us to solve tough problems that improve people’s daily lives. Let’s start, for instance, with something many people rely on every day: driving directions.

It’s pretty well known that our navigation apps rely on GPS satellites to pinpoint our location and complex algorithms to figure out the most direct route to get wherever we want to go. Most of the time, this works just fine. But what happens when there’s a traffic jam along the most direct route? In this case, following the algorithm—and the app’s directions—wouldn’t be ideal.

Fortunately, machine learning offers a solution to this problem.

What is machine learning, and how does it work?

Where traditional algorithms rely on human developers to provide explicit instructions, machine learning allows data to guide the process. Using a training set comprised of representative data, the computer develops a model that can be compared against an expected output. The more data the computer takes into account, the more accurate and sophisticated its model becomes without developers having to write additional lines of code.

The same advancements in storage and processing that have facilitated the rise of big data have also allowed machine learning to advance by leaps and bounds. Storage and processing frameworks like Hadoop and robust libraries for programming languages like R, Python, and Java have made it feasible for the first time to feed machine learning algorithms tremendous amounts of unstructured data. For data scientists, the ability to create algorithms that are guided by actual data is a huge advantage.

In the case of a navigation app, machine learning enables the app to improve its recommendations algorithm based on historical data from actual trips. This data can allow the algorithm to accurately predict whether a particular route is likely to be congested at certain times of day—whether it’s a major thoroughfare during rush hour, or the area around a major concert venue on the weekends—and recommend an alternative route if needed. At no point does a developer have to tell the computer there’s such a thing as “rush hour;” the computer will simply infer it from the data, and adjust its model accordingly.

What if the underlying source of the data changes? Say a major new highway is built someplace that completely changes the way people get around. In this case, our model would no longer be accurate. This is why it may be important to periodically retrain the algorithm with a more up-to-date dataset.

Keep in mind, creating an algorithm that takes data into account is not the same thing as creating an algorithm that evolves as more data passes through it. Depending on your business needs, you might consider incorporating a batch algorithm to periodically update your model, or a more advanced incremental algorithm that updates itself automatically as new data is added.

Making computers (and people) smarter

Machine learning’s ability to craft algorithms based on real-world datasets (especially large ones) has allowed developers and data scientists to solve problems that were previously considered unsolvable. For example, understanding natural language used to be one of the holy grails of computer science. The complexities and subtleties of human language made capturing it in an algorithm a daunting challenge. No matter how many rules and exceptions computer scientists and linguists added to their algorithms, the results were often awkward and unreliable.

This all changed with machine learning. Rather than giving the computer a static set of instructions (for example, in English the subject and verb must always agree in number), teams of data scientists deployed linguists to tag parts of speech in a training set of documents. Using those training sets, their computers were able to build models that inferred not just the formal rules of grammar and syntax, but also developed a robust tolerance for grammatical errors, spelling mistakes, etc. This technology now powers the grammar checks used by Microsoft Office and Google Docs, as well as more sophisticated tools like Google Translate.

Putting data to work

The techniques of machine learning can be applied across a wide array of industries, from finance to e-commerce to healthcare to streaming services. What all these applications have in common is that they rely on using real-world data, often in creative ways.

Here are just a few of the most common applications:

  1. Anomaly detection is the process of sifting through mountains of data and learning to identify instances that deviate from recognized patterns. This technology enables credit card companies to instantly identify potentially fraudulent transactions. It’s also used by email programs like Gmail and Outlook to identify and filter out spam.
  2. Natural language processing (NLP) enables computers to make sense of human language, whether written or spoken. Turning unstructured language into structured data allows aggregators like Google News to automatically sort news articles by subject. Consumer analytics companies use it to gauge customer satisfaction, and legal tech companies are able to create searchable, organized databases of legal opinions. When applied to human speech, NLP enables virtual assistants like Siri or Microsoft Cortana to understand and respond to spoken requests.
  3. Recommendation engines gather and analyze data about users’ behaviors and preferences to make predictions about what users might like based on their similarity to other users. Services as diverse as Amazon, Netflix, Spotify, and Tinder rely on recommendation engines to suggest new products, movies, music, and dates to their users.
  4. Named entity recognition is the ability to identify specific entities (like names, titles, actions, addresses, etc.) in documents. This technology allows Apple’s Messages app to automatically identify phone numbers and dates so that you can call a friend or save a date to your calendar with just a tap.
  5. Image recognition allows computers to extract meaning from still images and videos. This technology enables the U.S. Postal Service to decipher handwritten addresses and Facebook and other photo apps to detect and identify human faces. The startup Enlitic uses this technology to analyze medical images like X-rays, CT scans, and MRIs to spot trends and anomalies.

It’s still the early days for machine learning, and we’re only just now beginning to explore its full potential. Want to see how machine learning can change the way you do business? Search for freelancers with machine learning experience on Upwork today.

The Secret to Building a Team of Top-Notch Distributed Engineers

The Secret to Building a Team of Top-Notch Distributed Engineers

Download Now