data visualization homework

What Is Data Visualization: Brief Theory, Useful Tips and Awesome Examples

• Share on Facebook
• Share on Twitter

By Al Boicheva

in Insights , Inspiration

3 years ago

Viewed 10,396 times

Spread the word about this article:

Updated: June 23, 2022

To create data visualization in order to present your data is no longer just a nice to have skill. Now, the skill to effectively sort and communicate your data through charts is a must-have for any business in any field that deals with data. Data visualization helps businesses quickly make sense of complex data and start making decisions based on that data. This is why today we’ll talk about what is data visualization. We’ll discuss how and why does it work, what type of charts to choose in what cases, how to create effective charts, and, of course, end with beautiful examples.

So let’s jump right in. As usual, don’t hesitate to fast-travel to a particular section of your interest.

Article overview: 1. What Does Data Visualization Mean? 2. How Does it Work? 3. When to Use it? 4. Why Use it? 5. Types of Data Visualization 6. Data Visualization VS Infographics: 5 Main Differences 7. How to Create Effective Data Visualization?: 5 Useful Tips 8. Examples of Data Visualization

1. What is Data Visualization?

Data Visualization is a graphic representation of data that aims to communicate numerous heavy data in an efficient way that is easier to grasp and understand . In a way, data visualization is the mapping between the original data and graphic elements that determine how the attributes of these elements vary. The visualization is usually made by the use of charts, lines, or points, bars, and maps.

• Data Viz is a branch of Descriptive statistics but it requires both design, computer, and statistical skills.
• Aesthetics and functionality go hand in hand to communicate complex statistics in an intuitive way.
• Data Viz tools and technologies are essential for making data-driven decisions.
• It’s a fine balance between form and functionality.
• Every STEM field benefits from understanding data.

2. How Does it Work?

If we can see it, our brains can internalize and reflect on it. This is why it’s much easier and more effective to make sense of a chart and see trends than to read a massive document that would take a lot of time and focus to rationalize. We wouldn’t want to repeat the cliche that humans are visual creatures, but it’s a fact that visualization is much more effective and comprehensive.

In a way, we can say that data Viz is a form of storytelling with the purpose to help us make decisions based on data. Such data might include:

• Tracking sales
• Identifying trends
• Identifying changes
• Monitoring goals
• Monitoring results
• Combining data

3. When to Use it?

Data visualization is useful for companies that deal with lots of data on a daily basis. It’s essential to have your data and trends instantly visible. Better than scrolling through colossal spreadsheets. When the trends stand out instantly this also helps your clients or viewers to understand them instead of getting lost in the clutter of numbers.

With that being said, Data Viz is suitable for:

• Annual reports
• Presentations
• Social media micronarratives
• Informational brochures
• Trend-trafficking
• Candlestick chart for financial analysis
• Determining routes

Common cases when data visualization sees use are in sales, marketing, healthcare, science, finances, politics, and logistics.

4. Why Use it?

Short answer: decision making. Data Visualization comes with the undeniable benefits of quickly recognizing patterns and interpret data. More specifically, it is an invaluable tool to determine the following cases.

• Identifying correlations between the relationship of variables.
• Getting market insights about audience behavior.
• Determining value vs risk metrics.
• Monitoring trends over time.
• Examining rates and potential through frequency.
• Ability to react to changes.

5. Types of Data Visualization

As you probably already guessed, Data Viz is much more than simple pie charts and graphs styled in a visually appealing way. The methods that this branch uses to visualize statistics include a series of effective types.

Map visualization is a great method to analyze and display geographically related information and present it accurately via maps. This intuitive way aims to distribute data by region. Since maps can be 2D or 3D, static or dynamic, there are numerous combinations one can use in order to create a Data Viz map.

COVID-19 Spending Data Visualization POGO by George Railean

The most common ones, however, are:

• Regional Maps: Classic maps that display countries, cities, or districts. They often represent data in different colors for different characteristics in each region.
• Line Maps: They usually contain space and time and are ideal for routing, especially for driving or taxi routes in the area due to their analysis of specific scenes.
• Point Maps: These maps distribute data of geographic information. They are ideal for businesses to pinpoint the exact locations of their buildings in a region.
• Heat Maps: They indicate the weight of a geographical area based on a specific property. For example, a heat map may distribute the saturation of infected people by area.

Charts present data in the form of graphs, diagrams, and tables. They are often confused with graphs since graphs are indeed a subcategory of charts. However, there is a small difference: graphs show the mathematical relationship between groups of data and is only one of the chart methods to represent data.

Infographic Data Visualization by Madeline VanRemmen

With that out of the way, let’s talk about the most basic types of charts in data visualization.

They use a series of bars that illustrate data development.  They are ideal for lighter data and follow trends of no more than three variables or else, the bars become cluttered and hard to comprehend. Ideal for year-on-year comparisons and monthly breakdowns.

These familiar circular graphs divide data into portions. The bigger the slice, the bigger the portion. They are ideal for depicting sections of a whole and their sum must always be 100%. Avoid pie charts when you need to show data development over time or lack a value for any of the portions. Doughnut charts have the same use as pie charts.

They use a line or more than one lines that show development over time. It allows tracking multiple variables at the same time. A great example is tracking product sales by a brand over the years. Area charts have the same use as line charts.

Scatter Plot

These charts allow you to see patterns through data visualization. They have an x-axis and a y-axis for two different values. For example, if your x-axis contains information about car prices while the y-axis is about salaries, the positive or negative relationship will tell you about what a person’s car tells about their salary.

Unlike the charts we just discussed, tables show data in almost a raw format. They are ideal when your data is hard to present visually and aim to show specific numerical data that one is supposed to read rather than visualize.

Data Visualisation | To bee or not to bee by Aishwarya Anand Singh

For example, charts are perfect to display data about a particular illness over a time period in a particular area, but a table comes to better use when you also need to understand specifics such as causes, outcomes, relapses, a period of treatment, and so on.

6. Data Visualization VS Infographics

5 main differences.

They are not that different as both visually represent data. It is often you search for infographics and find images titled Data Visualization and the other way around. In many cases, however, these titles aren’t misleading. Why is that?

• Data visualization is made of just one element. It could be a map, a chart, or a table. Infographics , on the other hand, often include multiple Data Viz elements.
• Unlike data visualizations that can be simple or extremely complex and heavy, infographics are simple and target wider audiences. The latter is usually comprehensible even to people outside of the field of research the infographic represents.
• Interestingly enough, data Viz doesn’t offer narratives and conclusions, it’s a tool and basis for reaching those. While infographics, in most cases offer a story and a narrative. For example, a data visualization map may have the title “Air pollution saturation by region”, while an infographic with the same data would go “Areas A and B are the most polluted in Country C”.
• Data visualizations can be made in Excel or use other tools that automatically generate the design unless they are set for presentation or publishing. The aesthetics of infographics , however, are of great importance and the designs must be appealing to wider audiences.
• In terms of interaction, data visualizations often offer interactive charts, especially in an online form. Infographics, on the other hand, rarely have interaction and are usually static images.

While on topic, you could also be interested to check out these 50 engaging infographic examples that make complex data look great.

7. Tips to Create Effective Data Visualization

The process is naturally similar to creating Infographics and it revolves around understanding your data and audience. To be more precise, these are the main steps and best practices when it comes to preparing an effective visualization of data for your viewers to instantly understand.

1. Do Your Homework

Preparation is half the work already done. Before you even start visualizing data, you have to be sure you understand that data to the last detail.

Knowing your audience is undeniable another important part of the homework, as different audiences process information differently. Who are the people you’re visualizing data for? How do they process visual data? Is it enough to hand them a single pie chart or you’ll need a more in-depth visual report?

The third part of preparing is to determine exactly what you want to communicate to the audience. What kind of information you’re visualizing and does it reflect your goal?

And last, think about how much data you’ll be working with and take it into account.

2. Choose the Right Type of Chart

In a previous section, we listed the basic chart types that find use in data visualization. To determine best which one suits your work, there are a few things to consider.

• How many variables will you have in a chart?
• How many items will you place for each of your variables?
• What will be the relation between the values (time period, comparison, distributions, etc.)

With that being said, a pie chart would be ideal if you need to present what portions of a whole takes each item. For example, you can use it to showcase what percent of the market share takes a particular product. Pie charts, however, are unsuitable for distributions, comparisons, and following trends through time periods. Bar graphs, scatter plots,s and line graphs are much more effective in those cases.

Another example is how to use time in your charts. It’s way more accurate to use a horizontal axis because time should run left to right. It’s way more visually intuitive.

3. Sort your Data

Start with removing every piece of data that does not add value and is basically excess for the chart. Sometimes, you have to work with a huge amount of data which will inevitably make your chart pretty complex and hard to read. Don’t hesitate to split your information into two or more charts. If that won’t work for you, you could use highlights or change the entire type of chart with something that would fit better.

Tip: When you use bar charts and columns for comparison, sort the information in an ascending or a descending way by value instead of alphabetical order.

4. Use Colors to Your Advantage

In every form of visualization, colors are your best friend and the most powerful tool. They create contrasts, accents, and emphasis and lead the eye intuitively. Even here, color theory is important.

When you design your chart, make sure you don’t use more than 5 or 6 colors. Anything more than that will make your graph overwhelming and hard to read for your viewers. However, color intensity is a different thing that you can use to your advantage. For example, when you compare the same concept in different periods of time, you could sort your data from the lightest shade of your chosen color to its darker one. It creates a strong visual progression, proper to your timeline.

Things to consider when you choose colors:

• Different colors for different categories.
• A consistent color palette for all charts in a series that you will later compare.
• It’s appropriate to use color blind-friendly palettes.

5. Get Inspired

Always put your inspiration to work when you want to be at the top of your game. Look through examples, infographics, and other people’s work and see what works best for each type of data you need to implement.

This Twitter account Data Visualization Society is a great way to start. In the meantime, we’ll also handpick some amazing examples that will get you in the mood to start creating the visuals for your data.

8. Examples for Data Visualization

As another art form, Data Viz is a fertile ground for some amazing well-designed graphs that prove that data is beautiful. Now let’s check out some.

Dark Souls III Experience Data

We start with Meng Hsiao Wei’s personal project presenting his experience with playing Dark Souls 3. It’s a perfect example that infographics and data visualization are tools for personal designs as well. The research is pretty massive yet very professionally sorted into different types of charts for the different concepts. All data visualizations are made with the same color palette and look great in infographics.

My dark souls 3 playing data by Meng Hsiao Wei

Greatest Movies of all Time

Katie Silver has compiled a list of the 100 greatest movies of all time based on critics and crowd reviews. The visualization shows key data points for every movie such as year of release, oscar nominations and wins, budget, gross, IMDB score, genre, filming location, setting of the film, and production studio. All movies are ordered by the release date.

100 Greatest Movies Data Visualization by Katie Silver

The Most Violent Cities

Federica Fragapane shows data for the 50 most violent cities in the world in 2017. The items are arranged on a vertical axis based on population and ordered along the horizontal axis according to the homicide rate.

The Most Violent Cities by Federica Fragapane

Family Businesses as Data

These data visualizations and illustrations were made by Valerio Pellegrini for Perspectives Magazine. They show a pie chart with sector breakdown as well as a scatter plot for contribution for employment.

PERSPECTIVES MAGAZINE – Family Businesses by Valerio Pellegrini

Orbit Map of the Solar System

The map shows data on the orbits of more than 18000 asteroids in the solar system. Each asteroid is shown at its position on New Years’ Eve 1999, colored by type of asteroid.

An Orbit Map of the Solar System by Eleanor Lutz

The Semantics Of Headlines

Katja Flükiger has a take on how headlines tell the story. The data visualization aims to communicate how much is the selling influencing the telling. The project was completed at Maryland Institute College of Art to visualize references to immigration and color-coding the value judgments implied by word choice and context.

The Semantics of Headlines by Katja Flükiger

Moon and Earthquakes

This data visualization works on answering whether the moon is responsible for earthquakes. The chart features the time and intensity of earthquakes in response to the phase and orbit location of the moon.

Moon and Earthquakes by Aishwarya Anand Singh

Dawn of the Nanosats

The visualization shows the satellites launched from 2003 to 2015. The graph represents the type of institutions focused on projects as well as the nations that financed them. On the left, it is shown the number of launches per year and satellite applications.

WIRED UK – Dawn of the by Nanosats by Valerio Pellegrini

Final Words

Data visualization is not only a form of science but also a form of art. Its purpose is to help businesses in any field quickly make sense of complex data and start making decisions based on that data. To make your graphs efficient and easy to read, it’s all about knowing your data and audience. This way you’ll be able to choose the right type of chart and use visual techniques to your advantage.

You may also be interested in some of these related articles:

• Infographics for Marketing: How to Grab and Hold the Attention
• 12 Animated Infographics That Will Engage Your Mind from Start to Finish
• 50 Engaging Infographic Examples That Make Complex Ideas Look Great
• Good Color Combinations That Go Beyond Trends: Inspirational Examples and Ideas

Add some character to your visuals

Cartoon Characters, Design Bundles, Illustrations, Backgrounds and more...

Like us on Facebook

Subscribe to our newsletter

Be the first to know what’s new in the world of graphic design and illustrations.

• [email protected]

Browse High Quality Vector Graphics

E.g.: businessman, lion, girl…

Related Articles

The best 23 photo editing software tools to edit like a pro, 24 effective landing page design examples that convert in 2022, the best online learning platforms in 2022, how to get custom illustrations for your project, 12 famous vector artists and their mind-blowing portfolios, check out our infographics bundle with 500+ infographic templates:, enjoyed this article.

Don’t forget to share!

• Comments (2)

Al Boicheva

Al is an illustrator at GraphicMama with out-of-the-box thinking and a passion for anything creative. In her free time, you will see her drooling over tattoo art, Manga, and horror movies.

Thousands of vector graphics for your projects.

Hey! You made it all the way to the bottom!

Here are some other articles we think you may like:

How to Find Good Infographic Topics That Will Get You the Right Audience

by Nikolay Kaloyanov

Can You Become a Graphic Designer Without a Design Degree in 2022?

by Iveta Pavlova

Inspiration

33 peculiar character design styles of the modern day, looking for design bundles or cartoon characters.

A source of high-quality vector graphics offering a huge variety of premade character designs, graphic design bundles, Adobe Character Animator puppets, and more.

CSE 163, Summer 2020: Homework 3: Data Analysis

In this assignment, you will apply what you've learned so far in a more extensive "real-world" dataset using more powerful features of the Pandas library. As in HW2, this dataset is provided in CSV format. We have cleaned up the data some, but you will need to handle more edge cases common to real-world datasets, including null cells to represent unknown information.

Note that there is no graded testing portion of this assignment. We still recommend writing tests to verify the correctness of the methods that you write in Part 0, but it will be difficult to write tests for Part 1 and 2. We've provided tips in those sections to help you gain confidence about the correctness of your solutions without writing formal test functions!

This assignment is supposed to introduce you to various parts of the data science process involving being able to answer questions about your data, how to visualize your data, and how to use your data to make predictions for new data. To help prepare for your final project, this assignment has been designed to be wide in scope so you can get practice with many different aspects of data analysis. While this assignment might look large because there are many parts, each individual part is relatively small.

Learning Objectives

After this homework, students will be able to:

• Work with basic Python data structures.
• Handle edge cases appropriately, including addressing missing values/data.
• Practice user-friendly error-handling.
• Read plotting library documentation and use example plotting code to figure out how to create more complex Seaborn plots.
• Train a machine learning model and use it to make a prediction about the future using the scikit-learn library.

Expectations

Here are some baseline expectations we expect you to meet:

Follow the course collaboration policies

If you are developing on Ed, all the files are there. The files included are:

• hw3-nces-ed-attainment.csv : A CSV file that contains data from the National Center for Education Statistics. This is described in more detail below.
• hw3.py : The file for you to put solutions to Part 0, Part 1, and Part 2. You are required to add a main method that parses the provided dataset and calls all of the functions you are to write for this homework.
• hw3-written.txt : The file for you to put your answers to the questions in Part 3.
• cse163_utils.py : Provides utility functions for this assignment. You probably don't need to use anything inside this file except importing it if you have a Mac (see comment in hw3.py )

If you are developing locally, you should navigate to Ed and in the assignment view open the file explorer (on the left). Once there, you can right-click to select the option to "Download All" to download a zip and open it as the project in Visual Studio Code.

The dataset you will be processing comes from the National Center for Education Statistics. You can find the original dataset here . We have cleaned it a bit to make it easier to process in the context of this assignment. You must use our provided CSV file in this assignment.

The original dataset is titled: Percentage of persons 25 to 29 years old with selected levels of educational attainment, by race/ethnicity and sex: Selected years, 1920 through 2018 . The cleaned version you will be working with has columns for Year, Sex, Educational Attainment, and race/ethnicity categories considered in the dataset. Note that not all columns will have data starting at 1920.

Our provided hw3-nces-ed-attainment.csv looks like: (⋮ represents omitted rows):

Column Descriptions

• Year: The year this row represents. Note there may be more than one row for the same year to show the percent breakdowns by sex.
• Sex: The sex of the students this row pertains to, one of "F" for female, "M" for male, or "A" for all students.
• Min degree: The degree this row pertains to. One of "high school", "associate's", "bachelor's", or "master's".
• Total: The total percent of students of the specified gender to reach at least the minimum level of educational attainment in this year.
• White / Black / Hispanic / Asian / Pacific Islander / American Indian or Alaska Native / Two or more races: The percent of students of this race and the specified gender to reach at least the minimum level of educational attainment in this year.

Interactive Development

When using data science libraries like pandas , seaborn , or scikit-learn it's extremely helpful to actually interact with the tools your using so you can have a better idea about the shape of your data. The preferred practice by people in industry is to use a Jupyter Notebook, like we have been in lecture, to play around with the dataset to help figure out how to answer the questions you want to answer. This is incredibly helpful when you're first learning a tool as you can actually experiment and get real-time feedback if the code you wrote does what you want.

We recommend that you try figuring out how to solve these problems in a Jupyter Notebook so you can actually interact with the data. We have made a Playground Jupyter Notebook for you that has the data uploaded. At the top-right of this page in Ed is a "Fork" button (looks like a fork in the road). This will make your own copy of this Notebook so you can run the code and experiment with anything there! When you open the Workspace, you should see a list of notebooks and CSV files. You can always access this launch page by clicking the Jupyter logo.

Part 0: Statistical Functions with Pandas

In this part of the homework, you will write code to perform various analytical operations on data parsed from a file.

Part 0 Expectations

• All functions for this part of the assignment should be written in hw3.py .
• For this part of the assignment, you may import and use the math and pandas modules, but you may not use any other imports to solve these problems.
• For all of the problems below, you should not use ANY loops or list/dictionary comprehensions. The goal of this part of the assignment is to use pandas as a tool to help answer questions about your dataset.

Problem 0: Parse data

In your main method, parse the data from the CSV file using pandas. Note that the file uses '---' as the entry to represent missing data. You do NOT need to anything fancy like set a datetime index.

The function to read a CSV file in pandas takes a parameter called na_values that takes a str to specify which values are NaN values in the file. It will replace all occurrences of those characters with NaN. You should specify this parameter to make sure the data parses correctly.

Problem 1: compare_bachelors_1980

What were the percentages for women vs. men having earned a Bachelor's Degree in 1980? Call this method compare_bachelors_1980 and return the result as a DataFrame with a row for men and a row for women with the columns "Sex" and "Total".

The index of the DataFrame is shown as the left-most column above.

Problem 2: top_2_2000s

What were the two most commonly awarded levels of educational attainment awarded between 2000-2010 (inclusive)? Use the mean percent over the years to compare the education levels in order to find the two largest. For this computation, you should use the rows for the 'A' sex. Call this method top_2_2000s and return a Series with the top two values (the index should be the degree names and the values should be the percent).

For example, assuming we have parsed hw3-nces-ed-attainment.csv and stored it in a variable called data , then top_2_2000s(data) will return the following Series (shows the index on the left, then the value on the right)

Hint: The Series class also has a method nlargest that behaves similarly to the one for the DataFrame , but does not take a column parameter (as Series objects don't have columns).

Our assert_equals only checks that floating point numbers are within 0.001 of each other, so your floats do not have to match exactly.

Optional: Why 0.001?

Whenever you work with floating point numbers, it is very likely you will run into imprecision of floating point arithmetic . You have probably run into this with your every day calculator! If you take 1, divide by 3, and then multiply by 3 again you could get something like 0.99999999 instead of 1 like you would expect.

This is due to the fact that there is only a finite number of bits to represent floats so we will at some point lose some precision. Below, we show some example Python expressions that give imprecise results.

Because of this, you can never safely check if one float is == to another. Instead, we only check that the numbers match within some small delta that is permissible by the application. We kind of arbitrarily chose 0.001, and if you need really high accuracy you would want to only allow for smaller deviations, but equality is never guaranteed.

Problem 3: percent_change_bachelors_2000s

What is the difference between total percent of bachelor's degrees received in 2000 as compared to 2010? Take a sex parameter so the client can specify 'M', 'F', or 'A' for evaluating. If a call does not specify the sex to evaluate, you should evaluate the percent change for all students (sex = ‘A’). Call this method percent_change_bachelors_2000s and return the difference (the percent in 2010 minus the percent in 2000) as a float.

For example, assuming we have parsed hw3-nces-ed-attainment.csv and stored it in a variable called data , then the call percent_change_bachelors_2000s(data) will return 2.599999999999998 . Our assert_equals only checks that floating point numbers are within 0.001 of each other, so your floats do not have to match exactly.

Hint: For this problem you will need to use the squeeze() function on a Series to get a single value from a Series of length 1.

Part 1: Plotting with Seaborn

Next, you will write functions to generate data visualizations using the Seaborn library. For each of the functions save the generated graph with the specified name. These methods should only take the pandas DataFrame as a parameter. For each problem, only drop rows that have missing data in the columns that are necessary for plotting that problem ( do not drop any additional rows ).

Part 1 Expectations

• When submitting on Ed, you DO NOT need to specify the absolute path (e.g. /home/FILE_NAME ) for the output file name. If you specify absolute paths for this assignment your code will not pass the tests!
• You will want to pass the parameter value bbox_inches='tight' to the call to savefig to make sure edges of the image look correct!
• For this part of the assignment, you may import the math , pandas , seaborn , and matplotlib modules, but you may not use any other imports to solve these problems.
• For all of the problems below, you should not use ANY loops or list/dictionary comprehensions.
• Do not use any of the other seaborn plotting functions for this assignment besides the ones we showed in the reference box below. For example, even though the documentation for relplot links to another method called scatterplot , you should not call scatterplot . Instead use relplot(..., kind='scatter') like we showed in class. This is not an issue of stylistic preference, but these functions behave slightly differently. If you use these other functions, your output might look different than the expected picture. You don't yet have the tools necessary to use scatterplot correctly! We will see these extra tools later in the quarter.

Part 1 Development Strategy

• Print your filtered DataFrame before creating the graph to ensure you’re selecting the correct data.
• Call the DataFrame describe() method to see some statistical information about the data you've selected. This can sometimes help you determine what to expect in your generated graph.
• Re-read the problem statement to make sure your generated graph is answering the correct question.
• Compare the data on your graph to the values in hw3-nces-ed-attainment.csv. For example, for problem 0 you could check that the generated line goes through the point (2005, 28.8) because of this row in the dataset: 2005,A,bachelor's,28.8,34.5,17.6,11.2,62.1,17.0,16.4,28.0

Seaborn Reference

Of all the libraries we will learn this quarter, Seaborn is by far the best documented. We want to give you experience reading real world documentation to learn how to use a library so we will not be providing a specialized cheat-sheet for this assignment. What we will do to make sure you don't have to look through pages and pages of documentation is link you to some key pages you might find helpful for this assignment; you do not have to use every page we link, so part of the challenge here is figuring out which of these pages you need. As a data scientist, a huge part of solving a problem is learning how to skim lots of documentation for a tool that you might be able to leverage to solve your problem.

We recommend to read the documentation in the following order:

• Start by skimming the examples to see the possible things the function can do. Don't spend too much time trying to figure out what the code is doing yet, but you can quickly look at it to see how much work is involved.
• Then read the top paragraph(s) that give a general overview of what the function does.
• Now that you have a better idea of what the function is doing, go look back at the examples and look at the code much more carefully. When you see an example like the one you want to generate, look carefully at the parameters it passes and go check the parameter list near the top for documentation on those parameters.
• It sometimes (but not always), helps to skim the other parameters in the list just so you have an idea what this function is capable of doing

As a reminder, you will want to refer to the lecture/section material to see the additional matplotlib calls you might need in order to display/save the plots. You'll also need to call the set function on seaborn to get everything set up initially.

Here are the seaborn functions you might need for this assignment:

• Bar/Violin Plot ( catplot )
• Plot a Discrete Distribution ( distplot ) or Continuous Distribution ( kdeplot )
• Scatter/Line Plot ( relplot )
• Linear Regression Plot ( regplot )
• Compare Two Variables ( jointplot )
• Heatmap ( heatmap )

Make sure you read the bullet point at the top of the page warning you to only use these functions!

Problem 0: Line Chart

Plot the total percentages of all people of bachelor's degree as minimal completion with a line chart over years. To select all people, you should filter to rows where sex is 'A'. Label the x-axis "Year", the y-axis "Percentage", and title the plot "Percentage Earning Bachelor's over Time". Name your method line_plot_bachelors and save your generated graph as line_plot_bachelors.png .

Problem 1: Bar Chart

Plot the total percentages of women, men, and total people with a minimum education of high school degrees in the year 2009. Label the x-axis "Sex", the y-axis "Percentage", and title the plot "Percentage Completed High School by Sex". Name your method bar_chart_high_school and save your generated graph as bar_chart_high_school.png .

Do you think this bar chart is an effective data visualization? Include your reasoning in hw3-written.txt as described in Part 3.

Problem 2: Custom Plot

Plot the results of how the percent of Hispanic individuals with degrees has changed between 1990 and 2010 (inclusive) for high school and bachelor's degrees with a chart of your choice. Make sure you label your axes with descriptive names and give a title to the graph. Name your method plot_hispanic_min_degree and save your visualization as plot_hispanic_min_degree.png .

Include a justification of your choice of data visualization in hw3-written.txt , as described in Part 3.

Part 2: Machine Learning using scikit-learn

Now you will be making a simple machine learning model for the provided education data using scikit-learn . Complete this in a function called fit_and_predict_degrees that takes the data as a parameter and returns the test mean squared error as a float. This may sound like a lot, so we've broken it down into steps for you:

• Filter the DataFrame to only include the columns for year, degree type, sex, and total.
• Do the following pre-processing: Drop rows that have missing data for just the columns we are using; do not drop any additional rows . Convert string values to their one-hot encoding. Split the columns as needed into input features and labels.
• Randomly split the dataset into 80% for training and 20% for testing.
• Train a decision tree regressor model to take in year, degree type, and sex to predict the percent of individuals of the specified sex to achieve that degree type in the specified year.
• Use your model to predict on the test set. Calculate the accuracy of your predictions using the mean squared error of the test dataset.

You do not need to anything fancy like find the optimal settings for parameters to maximize performance. We just want you to start simple and train a model from scratch! The reference below has all the methods you will need for this section!

scikit-learn Reference

You can find our reference sheet for machine learning with scikit-learn ScikitLearnReference . This reference sheet has information about general scikit-learn calls that are helpful, as well as how to train the tree models we talked about in class. At the top-right of this page in Ed is a "Fork" button (looks like a fork in the road). This will make your own copy of this Notebook so you can run the code and experiment with anything there! When you open the Workspace, you should see a list of notebooks and CSV files. You can always access this launch page by clikcing the Jupyter logo.

Part 2 Development Strategy

Like in Part 1, it can be difficult to write tests for this section. Machine Learning is all about uncertainty, and it's often difficult to write tests to know what is right. This requires diligence and making sure you are very careful with the method calls you make. To help you with this, we've provided some alternative ways to gain confidence in your result:

• Print your test y values and your predictions to compare them manually. They won't be exactly the same, but you should notice that they have some correlation. For example, I might be concerned if my test y values were [2, 755, …] and my predicted values were [1022, 5...] because they seem to not correlate at all.
• Calculate your mean squared error on your training data as well as your test data. The error should be lower on your training data than on your testing data.

Optional: ML for Time Series

Since this is technically time series data, we should point out that our method for assessing the model's accuracy is slightly wrong (but we will keep it simple for our HW). When working with time series, it is common to use the last rows for your test set rather than random sampling (assuming your data is sorted chronologically). The reason is when working with time series data in machine learning, it's common that our goal is to make a model to help predict the future. By randomly sampling a test set, we are assessing the model on its ability to predict in the past! This is because it might have trained on rows that came after some rows in the test set chronologically. However, this is not a task we particularly care that the model does well at. Instead, by using the last section of the dataset (the most recent in terms of time), we are now assessing its ability to predict into the future from the perspective of its training set.

Even though it's not the best approach to randomly sample here, we ask you to do it anyways. This is because random sampling is the most common method for all other data types.

Part 3: Written Responses

Review the source of the dataset here . For the following reflection questions consider the accuracy of data collected, and how it's used as a public dataset (e.g. presentation of data, publishing in media, etc.). All of your answers should be complete sentences and show thoughtful responses. "No" or "I don't know" or any response like that are not valid responses for any questions. There is not one particularly right answer to these questions, instead, we are looking to see you use your critical thinking and justify your answers!

• Do you think the bar chart from part 1b is an effective data visualization? Explain in 1-2 sentences why or why not.
• Why did you choose the type of plot that you did in part 1c? Explain in a few sentences why you chose this type of plot.
• Datasets can be biased. Bias in data means it might be skewed away from or portray a wrong picture of reality. The data might contain inaccuracies or the methods used to collect the data may have been flawed. Describe a possible bias present in this dataset and why it might have occurred. Your answer should be about 2 or 3 sentences long.

Context : Later in the quarter we will talk about ethics and data science. This question is supposed to be a warm-up to get you thinking about our responsibilities having this power to process data. We are not trying to train to misuse your powers for evil here! Most misuses of data analysis that result in ethical concerns happen unintentionally. As preparation to understand these unintentional consequences, we thought it would be a good exercise to think about a theoretical world where you would willingly try to misuse data.

Congrats! You just got an internship at Evil Corp! Your first task is to come up with an application or analysis that uses this dataset to do something unethical or nefarious. Describe a way that this dataset could be misused in some application or an analysis (potentially using the bias you identified for the last question). Regardless of what nefarious act you choose, evil still has rules: You need to justify why using the data in this is a misuse and why a regular person who is not evil (like you in the real world outside of this problem) would think using the data in this way would be wrong. There are no right answers here of what defines something as unethical, this is why you need to justify your answer! Your response should be 2 to 4 sentences long.

Turn your answers to these question in by writing them in hw3-written.txt and submitting them on Ed

Your submission will be evaluated on the following dimensions:

• Your solution correctly implements the described behaviors. You will have access to some tests when you turn in your assignment, but we will withhold other tests to test your solution when grading. All behavior we test is completely described by the problem specification or shown in an example.
• No method should modify its input parameters.
• Your main method in hw3.py must call every one of the methods you implemented in this assignment. There are no requirements on the format of the output, besides that it should save the files for Part 1 with the proper names specified in Part 1.
• We can run your hw3.py without it crashing or causing any errors or warnings.
• When we run your code, it should produce no errors or warnings.
• All files submitted pass flake8
• All program files should be written with good programming style. This means your code should satisfy the requirements within the CSE 163 Code Quality Guide .
• Any expectations on this page or the sub-pages for the assignment are met as well as all requirements for each of the problems are met.

Make sure you carefully read the bullets above as they may or may not change from assignment to assignment!

A note on allowed material

A lot of students have been asking questions like "Can I use this method or can I use this language feature in this class?". The general answer to this question is it depends on what you want to use, what the problem is asking you to do and if there are any restrictions that problem places on your solution.

There is no automatic deduction for using some advanced feature or using material that we have not covered in class yet, but if it violates the restrictions of the assignment, it is possible you will lose points. It's not possible for us to list out every possible thing you can't use on the assignment, but we can say for sure that you are safe to use anything we have covered in class so far as long as it meets what the specification asks and you are appropriately using it as we showed in class.

For example, some things that are probably okay to use even though we didn't cover them:

• Using the update method on the set class even though I didn't show it in lecture. It was clear we talked about sets and that you are allowed to use them on future assignments and if you found a method on them that does what you need, it's probably fine as long as it isn't violating some explicit restriction on that assignment.
• Using something like a ternary operator in Python. This doesn't make a problem any easier, it's just syntax.

For example, some things that are probably not okay to use:

• Importing some random library that can solve the problem we ask you to solve in one line.
• If the problem says "don't use a loop" to solve it, it would not be appropriate to use some advanced programming concept like recursion to "get around" that restriction.

These are not allowed because they might make the problem trivially easy or violate what the learning objective of the problem is.

You should think about what the spec is asking you to do and as long as you are meeting those requirements, we will award credit. If you are concerned that an advanced feature you want to use falls in that second category above and might cost you points, then you should just not use it! These problems are designed to be solvable with the material we have learned so far so it's entirely not necessary to go look up a bunch of advanced material to solve them.

tl;dr; We will not be answering every question of "Can I use X" or "Will I lose points if I use Y" because the general answer is "You are not forbidden from using anything as long as it meets the spec requirements. If you're unsure if it violates a spec restriction, don't use it and just stick to what we learned before the assignment was released."

This assignment is due by Thursday, July 23 at 23:59 (PDT) .

You should submit your finished hw3.py , and hw3-written.txt on Ed .

You may submit your assignment as many times as you want before the late cutoff (remember submitting after the due date will cost late days). Recall on Ed, you submit by pressing the "Mark" button. You are welcome to develop the assignment on Ed or develop locally and then upload to Ed before marking.

• 0: Course Prep
• Week 1.1: Course Introduction
• Week 1.2: Getting Started
• Week 1.3: Operations & Data Types
• Week 2: Functions & Packages
• Week 3: Creating functions
• Week 4.1: Conditionals
• Week 4.2: Testing & Debugging
• Week 5: Loops
• Week 6: Vectors
• Week 7: Strings
• Week 9: Introduction to Python
• Week 11-1: Data Analysis Prelude
• Week 11-2: Data Frames
• Week 12: Data Wrangling
• Week 13: Data Visualization
• Week 14: Reproducible Reporting
• Week 15: Monte Carlo Methods
• HW 1 - Getting Started
• HW 2 - Functions & Packages
• HW 3 - Creating Functions
• HW 4 - Conditionals & Testing
• HW 5 - Loops
• HW 6 - Vectors
• HW 7 - Strings
• HW 8 - Python
• HW 9 - Data Frames
• HW 10 - Data Wrangling
• HW 11 - Data Visualization
• HW 12 - Reproducible Reporting
• Getting Help
• Programming in R
• Visualizing Data
• RStudio Server
• Schedule a meeting w/Prof. Helveston
• Source files

Homework 11 - Data Visualization

Due : 30 November by 11:00 pm Weight : This assignment is worth 4% of your final grade. Purpose, Skills, & Knowledge : The purposes of this assignment are: To practice exploring and data frames in R using the dplyr library To practice generating plots using the ggplot2 library Assessment : Each question indicates the % of the assignment grade, summing to 100%. The credit for each question will be assigned as follows: 0% for not attempting a response. 50% for attempting the question but with major errors. 75% for attempting the question but with minor errors. 100% for correctly answering the question. Rules : This entire assignment is SOLO . You may not work with other classmates, though you may consult instructors for help.

1) Staying organized [5%]

Download and use this template for your assignment. Inside the “hw11” folder, open and edit the R script called “hw11.R” and fill out your name, GW Net ID, and the names of anyone you worked with on this assignment.

2) Choose and load some data [5%]

For this assignment, you will need to find a dataset of your choosing and create three summary visualizations. To keep things manageable, choose one of the following datasets from the following libraries. Note that to load any of these data frames, all you need to do is install and load the library.

3) Inspect your data [10%]

Once you’ve chosen a data set, open your hw11.R file and begin exploring the data (be sure to load the library that contains the dataset at the top of your file). Write some code in code chunks to preview and summarize the data frame using some of the methods we’ve used in class. You should be able to quickly get an understanding of what variables are included and their nature. Consider the following questions in your exploration (you don’t have to write out answers to these questions - just write code to help you answer them by previewing the data in different ways):

• What is the total size of the data frame?
• What type of data is each variable (numeric, character, logical, date)?
• Do any variables have missing values? Why might that be?
• For numeric variables, what are the min and max values?
• For character variables, what are the unique values in the variable?
• For date variables, what time period do the observations in these data frames span?

Do not brush this step off - the more thoroughly you inspect your dataset, the easier (and better) you data exploration will be. This will be absolutely critical for making your plots. Make sure you take the time to develop an understanding of the variables in your dataset as it is nearly impossible to imagine what different plots might be worth creating otherwise.

4) Make plots [50%]

Now that you have a basic understanding of the dataset, make some plots to explore the variables in the data and their potential relationships. You may use base R plotting functions or the ggplot2 library to make your figures, but you must make at least two different types of figures, including:

• A scatterplot of involving at least two variables.
• A bar chart involving at least one variable.

You can choose to plot whichever variables you wish, but you must be able to interpret the results of your plot.

5) Interpret your plots [15%]

Below the plot code for each of your plots, write a description and interpretation of your plot in a comment. Make sure you address at least the following questions:

• Describe what variables you are plotting and why.
• Describe the primary relationship / trend / information you hope the reader will gain from your visualization.

6) Save your plots [10%]

At the bottom of your hw11.R file, write code to save each of your three plots in the plots folder. Save them as .png files.

7) Submit your files on Blackboard [5%]

Create a zip file of all files in your R project folder for this assignment and submit the zip file on Blackboard by the due deadline.

IS445 - Data Viz - ACG/ACU

This is the course website for Data Visualization, instructed by Jill Naiman ([email protected]).

Below, you will find the materials for each week, as well as the syllabus that includes contact information and a course outline.

Lectures and Materials

Martin luther king day holiday, no classes, enjoy, introduction.

Lecture 1 - Class Introduction & Why we Visualize

The syllabus for the course, along with discussions about "what" visualizations are, and how to orient yourself in the course. What are some of the basics of how we interpret visualizations? How can we describe the process of making choices, understanding our audience, and so forth?

Example HW 1 Submission

Installation Instructions

Prep Notebook, Week 2

Import notebook for HW 1

In class week 2 notebook

GDP dataset

Stitch Image

Data Storage and Operations

overview concepts brain

Lecture 2 - Data storage & Operations, Image data

When we draw something on a screen, how do we represent that internally, and how is that translated into pixels? How are values transformed from 0's and 1's into values we can manipulate and understand?

Assignment help - TurnItIn

How to submit homework with the TurnItIn framework

Extras, Lecture 2

More examples of drawing images in 2D; binary representations

Prep Notebook, Week 3

In class, Week 3

Buildings dataset

Corgi in Hat

Types of Viz and Choosing Colors

Lecture 3 - Colors and Color maps, Types of viz

How do colors work? What are the different ways we can map colors to values? What should we keep in mind when doing this?

Prep Notebook, Week 4

In class notebook, Week 4

Palette Colors (palette_colors.py)

Michigan Depth Map (86Mb)

Brain Scan (72Mb)

Beginning interactivity

Lecture 4 - Widgets & Traitlets for Interactivity

We talk about the basics of using Traitlets and data binding in visualization.

In Class notebook

Prep Notebook, Week 5

Extra prep notebook

The UFO Sitings Dataset (13Mb)

Continuing interactivity with bqplot

Lecture 5 - Grammar of Graphics & bqplot

We introduce the basics of bqplot & how it relates to grammar of graphics

In class Notebook, Week 6 --> In class Notebook, Week 6 --> In class Notebook, Week 6

Prep Notebook, Week 6

Wealth of Nations Library (wealth_of_nations.py)

Wealth of Nations Data - nations.json

More with dashboards & Map Viz

Lecture 6 - Dashboards & Maps

Linking data a bit about maps (if we have time)

In Class Notebook, Week 7

Prep Notebook, Week 7, Part 1

Prep Notebook, Week 7, Part 2

State export utilities (states_utils.py)

US State abbreviations (us_state_abbrev.py)

Surgery Charges Dataset (37Mb)

Earthquake sensor data (59Mb)

Earthquake locations data (12Kb)

Maps, maps and more maps

Lecture 7 - Maps!

More about maps and their projections

In class notebook

Prep Notebook, Week 8, Part 1

Prep Notebook, Week 8, Part 2

State export data (8Kb)

Spring Break, no classes, enjoy!

Starboard and intro to javascript.

Lecture 8 - Choosing viz & Online viz platforms

Choosing what viz type to use, and an introduction to Vega/Vega-lite and Starboard

In Class Notebook, Week 10

Prep Starboard Notebook, Week 10

The GDP dataset (online)

The Mobility dataset (online)

Viz Audience; More Starboard, Javascript & Vega-lite (and maybe Idyll)

Lecture 10 - Viz audience, Final Project Info

Considerations of audience, review about final projects, more with vega lite & Starboard, Idyll

Idyll Installation Instructions

In Class Notebook, Week 11

Prep Starboard Notebook, Week 11

Finish up with Starboard, Intro to Idyll

Lecture 11 - Starboard, Publishing & more in Idyll

Publishing your viz, vega-lite in Idyll and a bit of d3.js

Lecture 11 extras - more with Vega-lite

More with vega-lite

In Class Notebook, Week 12

Prep Starboard Notebook, Week 12

Prep index.idyll file, Week 12

In class Idyll resources, Week 12

Corgis per country over time

Subset of full Corgi database

More with Idyll, Publishing Viz

Lecture 12 - Publishing & Validation

More about publishing and validation, and more of d3.js in Idyll

In class Idyll materials, Week 13

Prep Idyll materials, Week 13

Starting d3.js histogram example

More vega-lite idyll examples

A few more Idyll+d3 things, Starting SciViz

Lecture 13 - 3D graphics, Intro to SciViz

How your computer and the internet process 3D graphics. What is scientific visualization?

In class Idyll resources, Week 14

Prep resources, Week 14

In class jupyter notebook

Prep Notebook, Week 14, Part 1

Prep Notebook, Week 14, Part 2

Solver library (solverlibs.py)

Galaxy Particle Simulation files (77Mb)

Isolated Galaxy dataset (292 Mb)

Jeyll Intro Slides

Scientific Viz & Guest lecture from the Advanced Visualization Lab

Lecture 14 - Scientific Visualization, notes on final project

More about scientific visualization

Prep Notebook, Week 15

Network Visualization & Word clouds

Lecture 15 - WordClouds, networks, and final project

How to analyze text data, viz of networks, and where to go from here.

Prep Notebook, Week 16, Part 1

Prep Notebook, Week 16, Part 2

Text corpus from Othello

Broad facebook data

Major node data

Homework 01: Data visualization

Due: friday, february 5 11:59pm et.

• Gain proficiency in data visualization
• Apply principles of effective data visualization to a real dataset
• Practice the data science workflow using git and GitHub

Clone assignment repo and start new project

Accept and create your private repository of the assignment at https://classroom.github.com/a/jfx0Jzeq

Clone the repository and open a new project in RStudio. See the first lab and recent lectures for a reminder of the steps.

In this assignment, we will work with the tidyverse package.

Diamond prices

In this assignment, you will perform an investigation of diamond prices based on 1,000 diamonds. Build effective and well-labeled visualizations to answer the questions below. For each question, show your code and output, and write your answers in complete sentences.

All plots should follow the best visualization practices discussed in lecture. Plots should include an informative title, axes should be labeled, and careful consideration should be given to aesthetic choices.

In addition, code and narrative should not exceed an 80 character per line limit.

For this assignment you must have at least three commits and all of your code chunks must have meaningful names. You may choose when you want to make your commits.

We will only examine a subset of the data, so include the code below in a code chunk at the start of your R Markdown file.

How many rows are in the diamonds_subset dataset? How many columns?

Examine the documentation of the diamonds dataset by running ?diamonds in the console. What is the meaning of clarity ? What is the worst clarity ? What is the meaning of color ? What is the best color ? Note: we are investigating only a subset of the data (so 1,000 not over 50,000 diamonds).

Construct a scatterplot of price versus carat . Describe the relationship.

Color the points in the price versus carat scatterplot by the diamond’s color . Describe the relationship.

Add a geom_smooth() for each color and add the argument se = FALSE to omit the bands surrounding the smoothed fit.

Examine the relationship between price and carat by clarity , using a separate scatterplot for each clarity .

Create a bar chart showing all of the colors , with the count of diamonds on the y-axis.

Create a segmented bar chart showing one bar per color , each bar going from 0 - 1, with the fill determined by cut .

Create a segmented bar chart showing one bar per color , each bar going from 0 - 1, with the fill determined by price . Does this plot work? Why or why not?

Create side-by-side boxplots of price for each color and comment on the relationship. Then construct a violin plot using geom_violin() . What do the violin plots reveal that boxplots do not? What do boxplots reveal that violin plots do not?

Come up with a research question based on these data and write it down. Then, create an effective data visualization that answers the question and write a brief paragraph explaining how your visualization answers the question. Your plot should be substantially and noticeably different from the plots you created above. Do not simply switch variables or make a minor modification. Be creative and have fun!

Knit to PDF to create a PDF document. Stage and commit all remaining changes, and push your work to GitHub. Make sure all files are updated on your GitHub repo.

Only upload your PDF document to Gradescope. Before you submit the uploaded document, mark where each answer is to the exercises. If any answer spans multiple pages, then mark all pages. Associate the “Overall” section with the first page.

6.894 : Interactive Data Visualization

Assignment 1: visualization design.

In this assignment, you will design a visualization for a small data set and provide a rigorous rationale for your design choices. You should in theory be ready to explain the contribution of every pixel in the display. You are free to use any graphics or charting tool you please – including drafting it by hand. However, you may find it most instructive to create the chart from scratch using a graphics API of your choice.

(See Resources for a list of visualization tools.)

Data Set: U.S. Population, 1900 vs. 2000

Every 10 years, the census bureau documents the demographic make-up of the United States, influencing everything from congressional districting to social services. This dataset contains a high-level summary of census data for two years a century apart: 1900 and 2000. The data is a CSV (comma-separated values) file that describes the U.S. population in terms of year, reported sex (1: male, 2: female), age group (binned into 5 year segments from 0-4 years old up to 90+ years old), and the total count of people per group. There are 38 data points per year, for a total of 76 data points.

Dataset: CSV Source: U.S. Census Bureau via IPUMS

• Start by choosing a question you'd like a visualization to answer.
• Design a static visualization (i.e., a single image) that you believe effectively answers that question, and use the question as the title of your graphic.
• Provide a short write-up (no more than 4 paragraphs) describing your design.

While you must use the data set given, you are free to transform the data as you see fit. Such transforms may include (but are not limited to) log transformation, computing percentages or averages, grouping elements into new categories, or removing unnecessary variables or records. You are also free to incorporate external data as you see fit. Your chart image should be interpretable without recourse to your short write-up. Do not forget to include title, axis labels or legends as needed!

As different visualizations can emphasize different aspects of a data set, you should document what aspects of the data you are attempting to most effectively communicate. In short, what story are you trying to tell? Just as important, also note which aspects of the data might be obscured or down-played due to your visualization design.

In your write-up, you should provide a rigorous rationale for your design decisions. Document the visual encodings you used and why they are appropriate for the data and your specific question. These decisions include the choice of visualization type, size, color, scale, and other visual elements, as well as the use of sorting or other data transformations. How do these decisions facilitate effective communication?

The assignment score is out of a maximum of 10 points. Historically, the median score on this assignment has been 8.5. We will determine scores by judging both the soundness of your design and the quality of the write-up. We will also look for consideration of audience, message and intended task.

We will use the following rubric to grade your assignment. Note, rubric cells may not map exactly to specific point scores.

We will reward entries that go above and beyond the assignment requirements to produce effective graphics. Examples may include outstanding visual design, meaningful incorporation of external data to reveal important trends, demonstrating exceptional creativity, or effective annotations or other narrative devices.

Submission Details

This is an individual assignment. You may not work in groups. Your completed assignment is due on Wednesday 2/12, by noon .

Submit your assignment using this form . The form expects your visualization to be an image (either a .png or .jpg). Please make sure your image is sized for a reasonable viewing experience -- readers should not have to zoom or scroll in order to effectively view your submission!

Resubmissions. Resubmissions will be regraded by teaching staff, and you may earn back up to 50% of the points lost in the original submission. To resubmit this assignment, please use this form and follow the same submission process described above. Include a short 1 paragraph description summarizing the changes from the initial submission. Resubmissions without this summary will not be regraded. Resubmissions will be due by 11:59pm on Saturday, 2/29. Slack days may not be applied to extend the resubmission deadline. The teaching staff will only begin to regrade assignments once the Final Project phase begins, so please be patient.

• Due: 12pm, Wed 2/12
• The Dataset
• Submission Form

tableau.com is not available in your region.

Introduction to Materials Informatics

Data visualization ¶

Authors: Enze Chen and Mark Asta (University of California, Berkeley)

This is an interactive exercise, so you will want to click the and open the notebook in DataHub (or Colab for non-UCB students).

Learning objectives ¶

This notebook contains a series of exercises that introduces data visualization in Python such that, by the end of the lesson, you will be able to:

Describe what different visualizations are communicating and their pros and cons.

Evaluate when a particular type of visualization is suitable for a particular goal.

Use the matplotlib package to create basic visualizations in Python.

We will progress through most of this exercise together as a group and please interrupt any time you have a question. ❓

These exercises are grouped into the following sections:

Intro to data visualization

Intro to matplotlib

Numerical data

Categorical data

Statistical analyses

Intro to data visualization ¶

Back to top

Like all the previous sections, before we dive right in, let’s think about the following question:

Pause and reflect : Why is data visualization an important topic?

Given that this is a lesson on data visualization , it kinda makes sense to just dive right in and display something rather than say something . So here’s a plot that may look familiar:

Surprise! Turns out this plot was made in Python all along. 😜 While we kinda cheated by hiding the details in the above code, you will be able to make a plot just like this one by the end of this lesson.

Intro to matplotlib ¶

In Python, there are many packages for data visualization, each with their own merits, but the base package that many of them are based off of is matplotlib , a software that was originally developed by John Hunter (see John Hunter, Computing in Science Engineering , 2007 ) and is now used by thousands of scientists. It has, in many aspects, become the NumPy equivalent for plotting, and integrates quite well with NumPy arrays and pandas DataFrames.

You might read the name “matplotlib” and think, “Hey, that kinda sounds like MATLAB.” Well, you’d be right, since many aspects of matplotlib were first developed to emulate MATLAB (read Hunter’s article to learn more), and the Pyplot module was purposely designed to give a MATLAB-like way of plotting. This allows us to get high-quality graphics (like MATLAB) while obeying Python principles of object-oriented programming (OOP) and being free to use. ✨

To use Pyplot, we write the following:

In the first line, we import the Pyplot module under the plt alias, which is a community standard.

In the second line, we have our first IPython magic function , and this one specifically displays our plot outputs directly below the code cell (using the inline backend) instead of creating a pop-up window like when running .py files. This backend is only for Jupyter notebooks and it will not work if you include it in a .py file.

Create Figure and Axes objects ¶

Step 1 : There are many ways to create figures using Pyplot, but in true OOP fashion, the best practice is to create Figure and Axes objects using the plt.subplots() function like so:

The Figure ( fig variable) is the top-level container for all plot elements while the Axes ( ax variable) is the object for a particular plot (this is not the plural form of the word “axis”). plt.subplots() creates both of these objects, which we then assign to the variables on the left. Imagine this step as the IRL-equivalent of saying, “Before we can paint a painting, we must fetch a new canvas.”

Step 2 : To make a plot, we can call a method like ax.plot(x, y) directly using the Axes object — not the Figure object or the Pyplot module (unlike MATLAB). This standard method plots \(y\) vs. \(x\) points and, by default, connects the points with lines in the order that they’re plotted. Note the syntax for ax.plot() is a list (or array) of \(x\) -coordinates and then a list of \(y\) -coordinates, not a list of \((x,y)\) pairs.

Step 3 : Finally, and perhaps unintuitively, we have to call plt.show() to actually display the plot. Matplotlib can be quite literal! As we shall soon see, this is also what makes it shine. 💖

We’ve done too much talking, so let’s put this into practice!

Exercise: Write a few lines of code to plot \(y = x ^ 2\) for the first five positive integers ¶

Exercise: plot \(y = \sin(x)\) in the interval \(x \in [0, 2\pi]\) ¶.

Hint : Importing numpy is a good idea!

A few stylistic tips ¶

We’re off to a great start! Before we continue further with other plot types, let’s discuss a few important features that are currently missing from the above two plots.

Pause and reflect : What are some ways we can improve the above plots?

Adding informative labels ¶

Whether it’s a title, axes labels, or a legend, plot labels are extremely important! They can be easily added to an Axes object using built-in methods:

ax.set_title('title_str') : Adds a title to the plot.

ax.set_xlabel('xlabel_str') : Adds a label to the \(x\) -axis.

ax.set_ylabel('ylabel_str') : Adds a label to the \(y\) -axis.

ax.legend() : Adds a legend to the plot.

Note : All of these methods must be called before plt.show() .

The first three methods are relatively straightforward as the input argument becomes the label. If you know \(\LaTeX\) , it can be rendered if you include it in the string, where '$E_g$' will render as \(E_g\) like in the first plot.

The last method ax.legend() will display a legend with labels only for plot elements that have labels assigned to them. To assign a label to a plot element, add label='legend_str' in the input arguments to the ax.plot() method.

Adding more elements to the plot ¶

If you want to add more plot elements (lines, points, etc.) to the plot, that can easily be done with more calls to ax.plot() . These lines will be added with a different color in the color cycle and they will not replace previous lines. This is different behavior than in MATLAB, where you have to type hold on to put multiple curves on the same plot. Hopefully this is a welcome change. 🙂

Improving visibility ¶

Step 1 : Perhaps one of your first criticisms of the demo plots was that they’re too small! This can be easily fixed by adding figsize=(width, height) as an input argument to plt.subplots() , where width and height are numbers representing “inches.” We recommend enlarging your plots to (8, 6) to start with, and adjust based on personal preference.

Step 2 : Another improvement to plot visibility is increasing the thickness of your lines. To change this, add a linewidth=width parameter to your call to ax.plot() , where width is a number. We recommend a line width of at least 4 , which you can assign to a CONSTANT variable

Step “3” : Finally, you’ll notice that if you only increase the size of your figure, then the font size remains small. To increase the font size, you have to add fontsize=size to every function call that adds text to your plot. For example:

We recommend a font size of at least 20 . If this seems cumbersome, that’s because it is—we’ll discuss a better option in the next lesson.

Step 3.5 : This is very subtle, but you’ll notice that the \(x\) - and \(y\) -ticks are also labeled, but you never called a function to label them(!), which means their font needs to be increased in a different way. To do this, we use the ax.tick_params() method like so:

The documentation reveals that this method is quite flexible, but we can just use the other default options for now, which includes changing the label size for both \(x\) and \(y\) axes.

Exercise: plot \(y_1 = \sin(x)\) and \(y_2 = \cos(x)\) with these improvements below ¶

Numerical data ¶.

So far we’ve discussed how to make line plots, which give you the ability to plot curves, functions, etc., which already covers a lot! There is another, related type of plot of numerical data that we want to briefly discuss, and that is a scatter plot like the very first figure in this notebook. To create a scatter plot in matplotlib, we can use the ax.scatter(x, y) method akin to ax.plot(x, y) . The nice part of plotting in matplotlib is that we can swap out the plotting method but keep a lot of the other methods unchanged!

Hmm, while the points seem OK, we can probably do better. In matplotlib language, the individual plot points themselves are called markers , so there’s a couple of things we can do to customize the markers:

Step 1 : The most obvious thing is we need to increase the marker size . There is an s parameter that can be a single number or an array of the same length as the number of points to set the size of each individual marker. For scatter plots, we have to make this number pretty large (e.g., 100 ).

Step 2 : We can also change the marker color using the c parameter, which like the size, can be a single color or an array of colors for each marker. In terms of color choices, there are several options! You can use a named color , a cycler notation , or any hex code (see here for a color picker).

Step 3 : We can also change the marker style using the marker parameter, which takes a string symbol that represents the marker. A list of all possible marker styles can be found in the documentation . The default is 'o' .

Categorical data ¶

In addition to numerical data, you might also find yourself working with categorical data (counts of materials type, crystal structure, etc.), so we will now discuss a canonical plot for categorical data: the bar chart . To make a bar chart in matplotlib, we can use the ax.bar(x, height) method that plots bars of height height at the positions specified by x ( x can also just be the labels, and Pyplot will space them apart evenly).

If you want custom labels for your bars, you can set the tick_label parameter in the function call. For more options and examples, we encourage you to look at the linked documentation up above for inspiration!

Exercise: create a bar chart of the crystal structure data of the first 50 elements ¶

Remember the data from the first day that mapped element \(\rightarrow\) structure? Recall that the file is located at '../../assets/data/week_1/01/element_structure.txt' and is structured as follows:

We’ll now create a bar chart that counts how many elements belong to each family. Write some code below to load in the data and then create a bar chart. Don’t forget to add some descriptive labels! 😊

🐼. Don’t forget about the power of sep !

Is there a way for us to count the number of occurrences of each value in a DataFrame?

Recall that we can get all the row labels of a Series/DataFrame with the .index attribute.

Additionally, you may have noticed that the tick labels are running into each other, which makes it pretty unreadable. There are many ways to fix this, but we’ll suggest two here:

You can rotate the labels by adding the following code:

Or you can plot a horizontal bar chart using the ax.barh(x, height) method. 😎

Statistical analyses ¶

For our final act, we’ll show you two more types of plots ( there are many more ) that can be generated using matplotlib that may be useful for your self-directed research and future work.

Histogram ¶

The first is a histogram , which can be very helpful for visualizing the distribution of your data. A histogram bins your data into different value ranges and plots the number of data points that fall within each range. We can plot a histogram using the ax.hist(x, bins=None) method, which takes in an array of values x and returns

vals : The values of the histogram bins, stored in an array of length nbins .

bin_edges : The edges of the bins, stored in an array of length nbins + 1 .

patches : Container of indiviudal artists used to create the histogram.

If you don’t need any of these outputs, then there’s no need to save them. We’ll plot a histogram for you below on the diffusion data from the first day, where we plot the distribution of distances from the origin at the final time step.

Saving your work! ¶

You may have noticed that in the last line above we snuck in the fig.savefig() method to demonstrate its usage. Recall that a Figure includes all plot elements, which is why savefig() is called from that object as opposed to the Axes object (one Figure can have multiple Axes , as we’ll show in the next lesson). This method is quite powerful, but for most cases, you will only need the following four arguments:

fname : The first argument should be the path to the filename you want to save it as, including the extension . Possible extensions include eps , jpg , png , pdf , svg , tif , among others.

dpi : This is short for “dots per inch,” and controls the resolution of your figure, more or less. We recommend a DPI of at least 300 for all plots, which is the minimum enforced by top-tier scientific journals.

bbox_inches : This setting controls the bounding box for your figure. tight is generally a good setting for auto-adjustment.

transparent : This determines whether Axes patches (background) will be transparent or not. False means a solid white background, while True might be nice if your figure is placed in a setting with a colored background.

Heatmaps are very useful for quickly identifying associations between pairs of variables arranged on a grid. These associations can be many things, and the one we will focus on in this example is the Pearson correlation coefficient ( \(r\) ). This statistical quantity measures the linear correlation between two sets of data, computed as the covariance of two variables divided by the product of their standard deviations . Mathematically, for two random variables \(X\) and \(Y\) , this is expressed as:

For our case when we’re dealing with a sample of data points given as \(\{(x_1, y_1), \dots, (x_n, y_n)\}\) , it is given by:

Visually, the Pearson correlation coefficient demonstrates the following behaviors (the line is fictitious and meant to help guide the eye):

pandas to the rescue! ¶

When doing exploratory data analysis (EDA) for your data, it is a very good idea to visualize the correlations between different variables as a heatmap to see if there are any expected or unexpected trends. Luckily for us, we do not have to compute Eq. 1 by hand, which would be very annoying when we’re trying to compute pairwise correlations for every possible pair between dozens of variables. Instead, we can take advantage of a built-in method from pandas, df.corr() , which computes the pairwise correlation of all the numerical columns and returns a matrix of correlations.

We can then use matplotlib to create our heatmap using the ax.imshow(X) method, where X is a matrix of values. With some clever styling, we can even get labels and numbers to appear. We’ll demonstrate this below with some elemental data that we saw previously.

Exercise: hmm… it’s not very clear what values the colors correspond to. Can you add a colorbar? ¶

What correlations do we see?

Conclusion ¶

This concludes our introduction to data visualization! 📊📈 For a 1-page summary of this information and more, you can check out the cheatsheet made by the developers of matplotlib.

Up next, we’ll discuss some common pitfalls and best practices when it comes to making visualizations. In the meantime, feel free to explore more on your own and let us know if you have any questions.

• Biostatistics
• Data Science
• Programming
• Social Science
• Certificates
• Undergraduate
• For Businesses
• FAQs and Knowledge Base
• Test Yourself
• Instructors

Interactive Data Visualization with Tableau

This course will teach you how to estimate descriptive quantities and sampling variances from complex surveys, and also how to fit linear and logistic regression models to complex sample survey data.

In this course you will learn about the interactive exploration of data, and how it is achieved using state-of-the-art data visualization software. You will learn to explore a range of different data types and structures, and about various interactive techniques for manipulating and examining data to produce effective visualizations. The learning process is hands-on as students are guided through an analysis of quantitative business data to discern meaningful patterns, trends, relationships, and exceptions that reveal business performance, potential problems and opportunities.

• Introductory, Intermediate
• Expert Instructor
• Tuiton-Back Guarantee
• 100% Online

Learning Outcomes

Students who complete this course will be able to:

• Apply principles of perception to data visualization
• Use software tools to interactively visualize relationships among variables
• Analyze distributions of data visually
• Use a range of displays to explore data
• Use parallel coordinate plots, scatterplots, and trellising to analyze multivariate data
• Visualize hierarchical data with treemap

Who Should Take This Course

Statistical analysts and data miners who need to explore and graph multivariate data, either to form impressions of the data or as a preliminary step to performing statistical tests or building models.

Our Instructors

Ms. madhuri maddipatla.

Madhuri Maddipatla is an analytics specialist and problem solver with 10+ years of experience in analytics consulting across multiple domains, including Retail, Consumer Packaged Goods, Healthcare, Finance, Manufacturing, and E-commerce.  Currently a Specialist with McKinsey and Company, she has been an instructor and mentor in the data analytics, data visualization and business consulting space for 6+ years now. She completed her M.S. in Data Science and Business Analytics at the University of North Carolina at Charlotte and worked on several analytics efforts with the industry and in the academic setup. She won several online crowd sourcing analytics contests and is a passionate problem solver and data science mentor.

Course Syllabus

• Information visualization characterization and history
• Elements of visual perception
• Software introduction and data preparation (merging data, getting started, export)
• Interaction techniques
• Distribution analysis
• Hands-on visual exploration of business data
• Time Series
• Multivariate views (scatterplots, parallel coordinate plots, trellising)
• Treemaps for hierarchical data
• Specialized visualizations
• Video demonstrations of novel techniques
• From visualization to visual analytics

Class Dates

Prerequisites.

The Statistics.com courses have helped me a lot, pushing me to the limit and making me learn much more than I expected I could. The knowledge I gained I could immediately leverage in my job … then eventually led to landing a job in my dream company – Amazon.

Karolis Urbonas

This program has been a life and work game changer for me. Within 2 weeks of taking this class, I was able to produce far more than I ever had before.

The material covered in the Analytics for Data Science Certificate will be indispensable in my work. I can’t wait to take other courses. Great work!

Stephen McAllister

I learned more in the past 6 weeks than I did taking a full semester of statistics in college, and 10 weeks of statistics in graduate school. Seriously.

Amir Aminimanizani

This is the best online course I have ever taken. Very well prepared. Covers a lot of real-life problems. Good job, thank you very much!

The more courses I take at Statistics.com, the more appreciation I have for the smart approach, quality of instructors, assistants, admin and program. Well done!

Leonardo Nagata

This course greatly benefited me because I am interested in working in AI. It has given me solid foundational knowledge…After completing this last course, I feel I have gained valuable skills that will enhance my employability in Data Science, opening up diverse career opportunities.

Richard Jackson

Frequently asked questions, what is your satisfaction guarantee and how does it work.

We offer a “Student Satisfaction Guarantee​” that includes a tuition-back guarantee, so go ahead and take our courses risk free. That’s our commitment to student satisfaction. Students may cancel, transfer, or withdraw from a course under certain conditions. If you’re not satisfied with a course, you may withdraw from the course and receive a tuition refund.

Please see our  knowledge center for more information.

Can I transfer or withdraw from a course?

We have a flexible transfer and withdrawal policy that recognizes circumstances may arise to prevent you from taking a course as planned. You may transfer or withdraw from a course under certain conditions.

• Students are entitled to a full refund if a course they are registered for is canceled.
• You can transfer your tuition to another course at any time prior to the course start date or the drop date, however a transfer is not permitted after the drop date.
• Withdrawals on or after the first day of class are entitled to a percentage refund of tuition.

Please see  this page for more information.

Who are the instructors at Statistics.com?

Statistics.com has more than 60 instructors who are recruited based on their expertise in various areas in statistics. Our faculty members are:

• Authors of well-regarded texts in their area;
• Advisory board members;
• Senior faculty; and
• Educators who have made important contributions to the field of statistics or online education in statistics.

The majority of our instructors have more than five years of teaching experience online at the Institute.

Please visit our faculty page for more information on each instructor at Statistics.com.

Please see our  knowledge center  for more information.

Visit our knowledge base and learn more.

Register For This Course

Related courses, visualization in r with ggplot2.

• Skill: Introductory, Intermediate
• Credit Options: ACE, CAP, CEU

Additional Information

Organization of course.

This course takes place online at The Institute for 4 weeks. During each course week, you participate at times of your own choosing – there are no set times when you must be online. Course participants will be given access to a private discussion board. In class discussions led by the instructor, you can post questions, seek clarification, and interact with your fellow students and the instructor.

At the beginning of each week, you receive the relevant material, in addition to answers to exercises from the previous session. During the week, you are expected to go over the course materials, work through exercises, and submit answers. Discussion among participants is encouraged. The instructor will provide answers and comments, and at the end of the week, you will receive individual feedback on your homework answers.

Time Requirements

This is a 4-week course requiring 10-15 hours per week of review and study, at times of your choosing.

Homework in this course consists of guided exercises using state of the art software.

In addition to assigned readings, this course also has an end of course data modeling project, and example software files.

Course Text

A recommended text for this course is Now You See It: Simple Visualization Techniques for Quantitative Analysis by Stephen Few.  Note: This text is not available in digital format. For those residing outside the US and not able to purchase this text, you may use The Truthful Art by Albert Cairo instead.

The use of Tableau software is illustrated and access to this program will be provided in the first lesson. Prior experience with Tableau is not expected or required.

Some students also use Spotfire, but it is not available as part of the course.  Want to use R?  Please see our course: Visualization in R with ggplot2 .

Course Fee & Information

Enrollment Courses may fill up at any time and registrations are processed in the order in which they are received. Your registration will be confirmed for the first available course date unless you specify otherwise.

Transfers and Withdrawals We have flexible policies to transfer to another course or withdraw if necessary.

Group Rates Contact us  to get information on group rates.

Discounts Academic affiliation?  In most courses you are eligible for a discount at checkout.

New to Statistics.com?  Click  here  for a special introductory discount code.

Invoice or Purchase Order Add $50 service fee if you require a prior invoice, or if you need to submit a purchase order or voucher, pay by wire transfer or EFT, or refund and reprocess a prior payment.

Options for Credit and Recognition

This course is eligible for the following credit and recognition options:

No Credit You may take this course without pursuing credit or a record of completion.

Mastery or Certificate Program Credit If you are enrolled in mastery or certificate program that requires demonstration of proficiency in this subject, your course work may be assessed for a grade.

CEUs and Proof of Completion If you require a “Record of Course Completion” along with professional development credit in the form of Continuing Education Units (CEU’s), upon successfully completing the course, CEU’s and a record of course completion will be issued by The Institute upon your request.

INFORMS-CAP This course is recognized by the Institute for Operations Research and the Management Sciences (INFORMS) as helpful preparation for the Certified Analytics Professional (CAP®) exam and can help CAP® analysts accrue Professional Development Units to maintain their certification.

This course has been evaluated by the American Council on Education (ACE) and is recommended for the upper-division baccalaureate degree, 2 semester hours in computer science, computer science systems, or information technology. Please note that the decision to accept specific credit recommendations is up to the academic institution accepting the credit.

Supplemental Information

Literacy, accessibility, and dyslexia.

At Statistics.com, we aim to provide a learning environment suitable for everyone. To help you get the most out of your learning experience, we have researched and tested several assistance tools. For students with dyslexia, colorblindness, or reading difficulties, we recommend the following web browser add-ons and extensions:

•   Color Enhancer  (for colorblindness)
• HelperBird  (for colorblindness, dyslexia, and reading difficulties)
• Mobile Dyslexic
• Color Vision Simulation  (native accessibility feature)
• Other native accessibility  features instructions
• Navidys  (for colorblindness, dyslexia, and reading difficulties)
• HelperBird for Safari  (for colorblindness, dyslexia, and reading difficulties)

Miscellaneous

Teaching Data Visualization: An Introduction

[cs_content][cs_section parallax=”false” style=”margin: 0px;padding: 45px 0px;”][cs_row inner_container=”true” marginless_columns=”false” style=”margin: 0px auto;padding: 0px;”][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/1″ style=”padding: 0px;”][cs_text]Team Data Visualization is proud to present a set of lesson plans that are ready to use in your classroom (networked or not). Whether you’ve been thinking about introducing a data visualization lesson of some kind, or have no idea what that would even look like or how it would fit in a writing classroom (or any classroom for that matter), we’ve got you covered.

Teaching in a DWRL classroom? Need a hand or want to schedule a data visualization workshop for your students? Visit our mentoring office in PAR 8B or email a staff member to learn more about our consulting and support services.

Featured image: A word cloud made with the 200 most common words in this post. [/cs_text][/cs_column][/cs_row][/cs_section][cs_section parallax=”false” style=”margin: 0px;padding: 45px 0px;”][cs_row inner_container=”true” marginless_columns=”false” style=”margin: 0px auto;padding: 0px;”][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/3″ style=”padding: 0px;”][x_button size=”global” block=”false” circle=”false” icon_only=”false” href=”https://www.dwrl.utexas.edu/2016/10/31/database-rhetorics/” title=”” target=”” info=”none” info_place=”top” info_trigger=”hover” info_content=””]Database Rhetorics[/x_button][/cs_column][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/3″ style=”padding: 0px;”][x_button size=”global” block=”false” circle=”false” icon_only=”false” href=”https://www.dwrl.utexas.edu/2016/10/31/lesson-plan-navigating-research-with-mindmup/” title=”” target=”” info=”none” info_place=”top” info_trigger=”hover” info_content=””]Navigating Research[/x_button][/cs_column][cs_column fade=”false” fade_animation=”in” fade_animation_offset=”45px” fade_duration=”750″ type=”1/3″ style=”padding: 0px;”][x_button size=”global” block=”false” circle=”false” icon_only=”false” href=”https://www.dwrl.utexas.edu/2016/10/31/infographic-recomposition/” title=”” target=”” info=”none” info_place=”top” info_trigger=”hover” info_content=””]Visual Literacy[/x_button][/cs_column][/cs_row][/cs_section][/cs_content]

Related Posts

Discuss This!: Structuring Reading Discussions through Collaborative Annotations

Proposal assignment.

Student Essay AI Co-Writing Public Demonstration

Flash Fellowship: The Re-lineator

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

• Show All Code
• Hide All Code
• Download Rmd

Data Visualization and Data Technologies

Recommended text books, prerequisites, first steps: do this today, getting set up, life expectancy in the americas in 2007, yearly snowfall in iowa city, internet adoption across the world, iowa wind turbines, iowa population in 2010, reproducible reports and analyses, tools for reproducibility, code for the iowa city snowfall example, code for the internet example.

Learn how to effecitvely use visualization for

• exploring and understanding data
• communicating and explaining insights

Learn how to use data technologies for

• acquiring data
• cleaning data
• organizing data

Learn how to do this in ways that are

• reproducible

Data visualization

• some history of visualization
• learning the basic graph types
• how to create basic graphs in R
• human perception, and how it affects visualization
• using understanding of perception to guide evaluation and design
• dynamic and interactive visualizations

Data technologies

• basic data types
• reshaping and transforming data
• aggregating and summarizing data
• merging several data sets
• regular expressions for cleaning data
• harvesting data from the web

Reproducible research and collaboration

• literate programming and data analysis
• version control for collaboration

Kieran Healy (2018) Data Visualization: A practical introduction , Princeton

Paul Murrell (2009). Introduction to Data Technologies , Chapman & Hall/CRC.

Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund (2023), R for Data Science (2nd Edition) , O’Reilly.

Claus O. Wilke (2019) Fundamentals of Data Visualization , O’Reilly, Inc. ( Book source on GitHub ; supporting materials on GitHub )

An introductory statistics course.

A regression course.

Strongly recommended: Prior exposure to basic use of statistical programming software, such as R or SAS, as obtained from a regression course.

• Short quizzes will be posted on ICON after most lectures.
• Homework assignments will be due approximately once a week.
• You will typically submit your work by pushing it to your GitLab repository by 5:00 PM on the due date.
• Your homework solutions should be written as reports, using proper sentences and paragraphs to present your results.
• You will do a project developing a visual analysis of a data set of your choosing.
• You can work on your own or in a group of up to three students.
• Your project should represent about 10 hours of work for each student.
• A one page proposal for your project is due on Monday, March 18.
• A final report on your project is due on Friday, May 3.
• Your project may be shared with the class through the class web page.

Your grade will be based on quizzes (10%), homework (70%) and the project (20%).

We will be using

• R for computing and graphics
• R Markdown for creating reproducible reports.
• git and the UI GitLab service for revision control and submitting work.

You will need an editor or IDE; you can use

• RStudio for editing and more
• any other editor or IDE

To access these tools you can

• use the UI IDAS RStudio Notebook Server ,
• use the CLAS Linux systems via the FastX remote desktop ,
• or install your own on your computer

For help installing your own a good place to start is https://happygitwithr.com/

Visit the UI GitLab site at https://research-git.uiowa.edu and log in with your HawkID.

Make sure you can access the UI IDAS Rstudio Notebook Server with your HawkID and password.

The server is available at

https://notebooks.hpc.uiowa.edu/spring2024-stat-4580-0001/hub/home .

If you cannot log into the RStudio server, please let your TA or me know immediately.

Make sure you are able to log into the CLAS Linux systems with your HawkID and password.

• The easiest way is to use the FastX client at https://fastx.divms.uiowa.edu .
• If you cannot log into the CLAS workstations, please let your TA or me know immediately.

Look at the brief introduction to git or the beginning of https://happygitwithr.com to see what git is about and how to get started with it.

Make sure you have access to R and try someting like this:

The result is a plot that looks like this:

Log into the UI GitLab site at https://research-git.uiowa.edu to get your GitLab account activated.

Decide where you want to work:

• UI IDAS RStudio Notebook Server
• FastX for accessing the CLAS Linux systems via the web interface or the desktop client .
• Your own computer.

Setup needed for IDAS RStudio Server:

• If you are registered then you should have an account now. If you add the course late you should have an account within a day.

Introduce yourself to Git .

Setup needed for CLAS Linux:

• Install the desktop client if you want to use it. Otherwise, use the web interface .
• Your account will be set up automatically the first time you log in.

Setting up your own computer: (A good resource for help with this is https://happygitwithr.com ):

Install the current version of R.

• You might have older versions from other courses (e.g. from Anaconda ).
• You will need to add packages as we go along.

Install RStudio if you want to use it (highly recommended).

Install Git.

Even if you decide to use your own computer you should make sure you can use the RStudio server or CLAS systems as a backup.

Some Examples

The data is from the GapMinder project.

A dot plot :

A bar chart :

Another (bad!) bar chart :

We will look at:

How to create these views using code that makes them easily reproducible.

How to assess their advantages and disadvantages as visual representations of the data

A data set with more variables for more countries and years is available in the gapminder R package.

Data preparation steps:

Filter the larger data set down to the countries and year we want.

Select the country name and life expectancy variables.

We will look at how to carry out these steps with reproducible code.

How did the winter of 2018/9 compare to other years?

The data are available from a NOAA web serice API as a CSV file.

Read in the CSV file.

Reshape the data to have columns date , TMAX , TMIN , SNOW and PRCP .

Filter out bogus dates created by the original format.

Convert units to more standard (American) ones (e.g. milimeters to inches).

Code is available here .

An example from Wilke (2019) with World Bank data .

The data are available in several formats (CSV, XML, Excel).

Data preparation:

Read in the data.

Filter down to the countries we want.

Reshape to have columns country , year , and users .

Data is available from the U.S. Wind Turbine Database .

There are two data sets:

Shape information for drawing the map.

Data on individual wind turbines.

Match up the projection used for the map and location data.

Again there are two data sets:

Shape data for drawing the map.

County population data from the 2010 census.

Merge or join the population data with the shape data.

Reproducibility

Preparing a report on a data analysis project usually involves

reading the data

wrangling the data into usable form

visualizing, summarizing, and modeling

writing a report that includes your results

To make your work reproducible for someone else, or for you when the data changes, it is best to use code for the entire workflow.

R Markdown is one technology that supports this.

R Markdown files contain report text along with code to produce numerical and graphical results.

Tools are available to

convert an R Markdown file into a PDF or HTML report;

extract the code used to produce the computational and graphical results.

This page was generated from the R Markdown file intro.Rmd .

You will be creating R Markdown files like this for your homework and project.

Some R Markdown tutorials:

R Markdown: The Definitive Guide by Yihui Xie is a book-length presentation.

The R Markdown Home Page has a link to a tutorial .

Read the data:

Reshape from (very) wide to (too) long:

Extract the day as a number:

Reshape from too long to tidy with one row per day, keeping only the primary variables:

Add a date variable for plotting and to help get rid of bogus days:

Make units more standard (American):

Add a Month factor with abbreviated levels:

Associate January through June with the winter starting in the previous year:

Compute the winter totals and the total for the 2018/9 winter:

Create the histogram and show the 2018/9 total:

From code for the Visualizing amounts chapter in Claus Wilke’s Fundamentals of Data Visualization .

Reading in the data:

Reshape to longer format:

Select some countries to include in the plot:

Get ordering by 2017 levels:

The basic plot:

Adjust color palette and guide:

Adjust x and y scales:

Add layer for NA values:

Final plot with title and theme adjustments:

Already have an account? Login

Test prep and homework help from private online Data Visualization tutors

Our online Data Visualization tutors offer personalized, one-on-one learning to help you improve your grades, build your confidence, and achieve your academic goals.

Top 50 online Data Visualization tutors

15 years of tutoring

English , Urdu

Lahore , Pakistan

USD $ 25 /hr

NATIONAL COLLEGE OF BUSNIESS ADMINISTRATION AND ECONOMICS , VIRTUAL UNIVERSITY OF PAKISTAN

STATISTICS / R studio / SPSS /STATA / BUSINESS STATISTICS TUTOR

Hello! I am a University Level STATISTICS / R studio / SPSS /STATA / BUSINESS STATISTICS AND AP STATISTICS TUTOR who can teach all topics of statistics at all levels. I am having vast experience in teaching statistics on different online platforms. I am an expert in R studio from which you can perform an analysis of your research project/ thesis. If you need any help related to any topic of statistics feel free to schedule your first lesson with me I will be looking forward to you!

Subjects : AP Statistics, Actuarial Science, Biostatistics I, Biostatistics II, Business Analysis, Computational statistics, Data Analysis, Data Visualization, Econometrics, Quality Management, R Programming, Stata, Statistical Mechanics, Statistical Package for Social Sciences (SPSS), Statistics

5 years of tutoring

English , Malayalam

Toronto , Canada

CAD $ 45 /hr

Arizona State University , Model engineering college

Data Science professional, Business Analytics Master’s graduate and an enthusiastic tutor to help you achieve your career/academic goals

I am a tech professional with more than 8 years of industry experience in data analytics and data science and more than 5 years of tutoring experience. My area of expertise is Data Analytics, Data Science and Mathematics. I enjoy sharing my knowledge and helping students at various skill levels to up skill and level up in their careers. I engage with students from all academic and professional backgrounds to help them achieve their academic/ career goals. Every student is different and their needs different as well. With the right approach, individually tailored curriculum, practice and dedication to learn, I believe everyone can master analytics. If this is what you are looking for, you can book a risk-free FREE consultation with me to know more.

Subjects : AP Statistics, Applied Mathematics, Data Analysis, Data Science, Data Visualization, Database, Math, MySQL, SQL, SQL Server

2 years of tutoring

English , Marathi

Mississauga , Canada

CAD $ 18 /hr

D.G.Ruparel College of Arts , Science and Commerce , Lambton College

An enthusiastic person who is keen on solving difficulties and circulating precise knowledge.

Microsoft-certified analyst in the field of data visualization, and analysis. Being a Business analyst student, a lot of work is done utilizing Microsoft Excel, unravelling statistical difficulties, and composing queries. So I would be keen on circulating my skillset with others by solving their cases and instructing them on the relevant explanations.

Subjects : AP Statistics, Data Analysis, Data Management, Data Visualization, Marathi, Math/Science, Maths, Microsoft Excel, MySQL, Oracle SQL, Python, Statistics

Kiswahili , English

Nairobi , Kenya

USD $ 15 /hr

Strathmore University

Expert Data Scientist with Tutoring experience

Experienced data scientist with 6 years of experience. Seeking to increase data efficiency for Well-paying clients. Achievements include creating elastic net regression models to predict company stock prices with 25% more accuracy than the historical average. Achieved 20% improvement in investment returns across all clients. Highly skilled in machine learning, data visualization, statistics, problem-solving, and creative thinking. Beat 200+ statistics professors and big data professionals in an NCAA pool, by specifying models that fit the problem best using experimental and validation subsets.

Subjects : Biostatistics II, Computational statistics, Data Analysis, Data Mining, Data Science, Data Structures & Algorithms, Data Visualization, Essay Writing, Health and Medicine, Nursing, R Programming, Stata, Statistical Package for Social Sciences (SPSS), Writing

4 years of tutoring

English , Hindi , Bengali

Kolkata , India

USD $ 20 /hr

Savitribai Phule Pune University , Presidency University, Kolkata

A Geographer, Geomorphologist and GIS and Remote Sensing Analyst

I am a geographer with expertise in sub-disciplines of Geomorphology, Oceanography, and tectonic Geology. I am also an expert GIS and Remote Sensing Analyst. GIS and RS Softwares I predominantly use are: ESRI ArcGIS Desktop & ArcGIS Pro, MapInfo Pro, QGIS, Surfer, ENVI, SNAP, ERDAS, Geomatica. I also use DBMS and data Visualization softwares such as: Tableau, MS Excel, Grapher and Origin Pro.

Subjects : Earth Science, Environmental Science, Geology, Geomorphology, Human Geography, IELTS, Microsoft Excel, Microsoft Suite, Microsoft Word, Oceanography, Scientific Research, Scientific Writing, World Geography

Personalize your search. Find your perfect tutor today!

How it works

Private online tutoring in 3 easy steps

Find the best online tutor.

Discover a vast selection of online tutors who specialize in your course. Our online tutors cover all subjects and levels, so you can easily find the perfect match for your needs.

Book online sessions at any time

Schedule a session with your online tutor via desktop or mobile. Collaborate with your tutor and learn effectively in real-time.

Join our online classroom

Connect with your online tutor through our interactive online classroom. Share your course syllabus and create a customized plan for success.

Why TutorOcean

Expert help with the best online tutors

Our online tutors offer personalized, one-on-one learning to help you improve your grades, build your confidence, and achieve your academic goals.

Unified platform

Everything you need for successful online learning

Private tutors, interactive online classroom, pay as you go, online tutoring, explore thousands of online tutors. start learning now.

Success stories

Revolutionizing education with the power of online tutoring

“Akshay is an exceptional Pre-calculus tutor for university-level students. He has a great way of explaining complex concepts and ensures that his students understand them. He is always ready to provide additional explanations if needed. I highly recommend him and look forward to booking him again.” — Sasha

“Richard is an exceptional tutor who has the ability to explain complex concepts in a simplistic way. His step-by-step instructions help to build confidence and understand the material better. Furthermore, he provides numerous tips and resources to facilitate success.” — Jessica

“I had a session on Linear Algebra, and it was very helpful. Mirjana was excellent in explaining matrices, and I could understand the concepts quite well. I would definitely request her assistance again.” — Lateefah

“Students struggling in math should seek help from Reza. He is patient, composed, and adept at explaining complex concepts in a clear and understandable way. He is also very generous with his time and willing to assist students on short notice.” — Rajasiva

“Sierra provided me with an exceptional tutoring session in chemistry. She was patient and made sure that I fully comprehended every concept. I am grateful for her assistance.” — Erin

“Michael did an excellent job in assisting me to comprehend various types of isomers. His tips and tricks were beneficial in resolving challenging problems.” — Jada

“I have found Anisha to be an exceptionally patient tutor who provides clear explanations that have helped me to comprehend various topics. I would strongly recommend her to anyone who needs assistance.” — Sam

“I received invaluable assistance from Patrick in terms of the direction for my papers. Collaborating with him was a comfortable experience, and it made the writing process much more manageable.” — Stephanie

“Elena's assistance was invaluable to me during my college essay revision session on Greek Mythology for the Humanities subject. She provided positive and helpful feedback and demonstrated expertise in several areas, which she explained very nicely.” — Abigail

Frequently asked questions