Best Public Datasets for Machine Learning

Machine learning has been widely used in industry and academia to build computer programs that perform complex tasks, such as understanding speech or written language and translating languages, identifying faces or objects in images or videos, playing board games, or driving cars. The performance of machine learning systems greatly depends on the quality of their training data. There are many publicly available machine learning datasets that you can use to train your machine learning models. In this article, we will discuss  useful public machine learning datasets and how to use them to build successful machine learning applications. Today’s businesses are using machine learning to improve their operations and reduce costs, with the aim of getting more out of every employee, dollar, and customer interaction.

But many businesses don’t know where to start when it comes to data collection and analysis for machine learning purposes. With that in mind, we have put together this guide on the best public datasets for machine learning – and how to use them. Machine learning models are useless if they aren’t fed high-quality data in order to learn from, and there are plenty of free public machine learning datasets available to get you started on your machine learning projects. This article outlines the top best public datasets for machine learning based on their size, ease of access, type of data, and purpose of the dataset. It also goes over some ways to use these machine learning datasets, so you can immediately get started with your own machine learning projects!

1) Papers about Famous Datasets

Papers are always a good place to start if you want an overview of a dataset. They give you an idea of what kind of data is available, how it’s being used, and whether it has limitations. For example, looking at papers about MNIST will give you a feel for its strengths and weaknesses, as well as highlight some interesting trends in machine learning research. This doesn’t mean that you should limit yourself only to papers about machine learning datasets; plenty of great data science work is published alongside code and research on new techniques or features. But when starting out with new data, reading papers can be a great way to get ideas about how best to use it and potentially find new avenues of study. The following is a list of public machine learning datasets and the papers they were featured in –

1) ImageNet: ImageNet: A Large Visual Database for Recognizing Objects (1999)
– 2) WikiTree: WikiTree: A collaborative genealogy wiki (2006)
– 3) English Wikipedia: Wikipedia in November 2005 (2005)
– 4) Facebook Likes: Likes on Facebook Reflect World Popularity (2011)
– 5) Reddit comments: Statistical Relational Modeling of Reddit Conversations (2015)
– 6) Vine videos: Semantic Analysis of Vine Videos Using Rich Features from Audio and Video Channels (2016), Learning Simple Interests from Vine Videos via Automatic Tag Discovery (2016).

Enroll in our latest machine learning course in the Entri app

2) Stack Overflow Dataset

This dataset includes users, tags, and questions in Stack Overflow’s Q&A site, spanning from 2008-present. This data was released by a member of Stack Overflow’s data team as part of their open-data initiative. This is an incredibly rich dataset that can be used for all sorts of questions about how people ask questions about programming languages and how these trends have changed over time. Data specialists may also want to examine other tags such as popular tags and safe searches that are often overlooked because they’re more niche. Try experimenting with them to see what you can find! Here are a few fun datasets to try:
Miles Davis album covers
Reddit’s top posts
Google Map APIs usage

3) Large Tree Archive Dataset

The world’s largest collection of tree data covers about 11,000 different species and is available for download in a wide range of formats. The dataset is compiled from observations made by humans and satellites over several decades. It provides data on tree attributes such as height, diameter, habitat, and environment. These are variables that are fundamental to understanding forest ecology. Trees cover 30 percent of Earth’s surface and play an important role in sustaining life; they affect climate change by absorbing carbon dioxide while providing shelter and food. But trees also face destruction due to logging or other factors like drought or disease. As a result, it is crucial to have robust information about them if we are going to devise ways of conserving them properly in today’s rapidly changing world.

The Large Tree Archive Project can help scientists understand the patterns of how human activities impact forests and how their knowledge can be used to develop new strategies for sustainable management. With so many uses, this project could have a huge impact on the way forests are studied worldwide. For example, the dataset will be useful for researchers at Duke University who want to study the link between water pollution and water-borne pathogens. It will also be valuable to forestry experts at universities across Europe studying pest control or those looking at logging-related mortality rates in Swedish mountain pine beetle populations. Scientists involved with land-use decisions—like those deciding where roads should go—can also use this data when determining what areas should receive priority protection. Finally, municipal authorities can use these findings when planning sewage systems and watersheds around urban areas in order to minimize contamination.

4) General Purpose Image Recognition Benchmark Dataset

The ImageNet Large Scale Visual Recognition Challenge is an annual challenge sponsored by Microsoft, Google, and Facebook. It provides a large annotated dataset of labeled images that can be used for training image recognition algorithms. You can also download pre-trained models from several participating teams, allowing you to get up and running quickly. New challenges are released each year. There’s also a smaller, much less well-known competition with datasets from different object categories, called CLEVR (Contemporary Large-scale Environment for Visual Recognition). This is available year-round and a great source of datasets if your field has new types of content that are popping up regularly. If you’re interested in facial recognition then there’s FaceScrub which contains pictures of human faces in various poses.

5) Yahoo Open Directory Project Dataset

The Yahoo! Directory (Yahoo Open Directory Project, YODP) is a human-edited directory of websites. It was created by Yahoo in early 2003 as an alternative to human-edited search engines that generated listings based on popularity. The project shut down on July 18, 2011, due to low participation and a lack of interest in editing its web pages. There are nearly 1 billion pages in Yahoo!s Web directory, almost double the number from three years ago when it opened up its data. The dataset has been used by researchers working on better algorithms for searching through data and organizing information about people and businesses across the internet. Researchers have also studied the usefulness of other classifiers, including neural networks. They found that neural networks were able to learn concepts without much supervision from humans because they rely on human feedback only after they’ve made a classification decision.

Enroll in our latest machine learning course in the Entri app

They found that this makes them good candidates for exploring deep learning techniques such as unsupervised clustering and semi-supervised learning. In addition, Google uses YODP extensively to train their own products like Google Maps and News. These two datasets combined to make one of the best collections available on how popular places change over time and how many people know about those places. New York Times Opinion Poll: Every week, reporters at The New York Times ask readers four questions: What’s your general opinion of Barack Obama? What’s your general opinion of Mitt Romney? Who would you rather be president? Do you think America needs a third-party candidate who will shake things up? Responses are recorded anonymously so poll-takers can’t see what others say until they’ve answered all four questions themselves.

6) Labeled Faces in the Wild Datasets

Labeled Faces in the Wild (LFW) is a popular dataset that contains labeled face images taken from a set of popular social networking websites. It consists of 50,000 headshots randomly sampled from a pool of 4 million such images. These samples were annotated by humans with regard to whether two people shown in them are actually pictures of one and the same person or not. This was important since it allows one to identify different types or shapes of human faces simply by analyzing which training patterns they could be associated with. LFW is freely available online and available in multiple formats including JPEG, XML, PDF, and MATLAB. The project homepage also has links to data pre-processing scripts, analysis tools, and an annotated gallery of sample images. The collection can be downloaded from the LFW webpage at CMU and offers an interesting resource for learning how to develop face recognition algorithms. ImageNet: ImageNet is a database built as part of the Connected Layouts project hosted at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). Its main purpose is to be used as a benchmark in computer vision competitions through yearly challenges like ILSVRC – ImageNet Large Scale Visual Recognition Challenge.

7) MS-COCO Image-Labeling Facial Landmark Dataset

Our first entry is also one of my personal favorites. If you’re building a speech-to-text app, you’re probably interested in gathering as much data as possible. Librispeech is a collection of recordings from over 100 public domain books (including many classics), divided into 500,000 labeled segments. The most important thing about Librispeech is that it’s clean, standardized, and easily parsable. It’s not nearly as big as MUTYH or VCTK, but that’s exactly why I like it! Having fewer hours of audio per dataset means your algorithm will have more time to learn from each individual recording, giving you better results in less time. So if you’ve got the cash to invest, take a look at MS-COCO Image-Labeling Facial Landmark Dataset for any text classification needs. MS-COCO Image-Labeling Facial Landmark Dataset: One of my favorite things about MS-COCO is how easy it is to download the entire dataset from scratch without going through any intermediaries. With hundreds of thousands of images labeled with facial landmarks and emotions on both the left and right sides of the face, this dataset is perfect for tackling computer vision problems ranging from face detection to identifying expressions! The labeling includes just six emotions – anger, disgust, fear, happiness, sadness, and surprise – so keep that in mind when selecting datasets.

8) NHL Statistics API

Hockey is a game of streaks and outliers. If you’re using machine learning techniques in your analysis, it’s essential that you have access to large datasets that capture every play of every game so you can be sure not to miss anything. The National Hockey League provides a great example of such a dataset on its web portal. You can easily see how players move around the ice from minute to minute and how those patterns change over multiple seasons. It also includes shots, goals, and other key information about each team, which is useful for both training and testing algorithms. The API requires an API key (provided by entering an email address), but there are many different levels of access available based on need. All NHL data is collected through broadcast footage or games played at their arenas, and the league does not license any content from third parties. That means it offers complete transparency for what teams are doing with their rosters year-round, as well as when they’re playing home or away games. The API lets you select specific games or quarters within a game as well as which season you want to study; then feeds back statistics like goals scored and penalties taken. All data comes directly from the NHL database after being processed by VizQL.

9) Librispeech Audiobook Review Dataset

LibriSpeech is a large collection of public domain audiobooks that are divided into speech segments, each of approximately two seconds in length. In total, there are 175 hours of audio data available in LibriSpeech. The LibriSpeech corpus is a non-traditional dataset; meaning it doesn’t fit any standard machine learning framework and requires adaptation on behalf of your model before it can be used effectively as input. If you don’t mind spending a little extra time preprocessing then Librispeech is one of the best datasets for practicing your extraction skills. __ We recommend using this dataset with TensorFlow,__ an open-source software library for numerical computation using data flow graphs. The Google team has designed TensorFlow to suit both new users without much experience in computer programming, as well as experts looking for highly flexible models. This flexibility comes at a cost, however: the setup of models with TensorFlow takes longer than with other libraries such as Scikit-Learn or Torch (or even sci-kit-learn & Torch). If you need speed over flexibility we recommend exploring PyTorch or Caffe2 instead.

Conclusion

We hope you’ve found some interesting datasets and examples of machine learning techniques. There are much more data sets out there that I didn’t have time to include, so if you find any others please leave a comment with a link. Thanks! Good luck with your machine learning adventures! *One final word of caution: it’s worth noting that not all publicly available datasets are created equal. We have covered here will give you an excellent starting point but be sure to do some research before choosing the one for your project as there is often a trade-off between the size and depth of the dataset. If you are interested to learn new coding skills, the Entri app will help you to acquire them very easily. Entri app is following a structural study plan so that the students can learn very easily. If you don’t have a coding background, it won’t be any problem. You can download the Entri app from the google play store and enroll in your favorite course.