data 8 homework solutions github

Data 8: Foundations of Data Science

InclusionBridge, Bridge to Data Fundamentals 23-24

Lecture Zoom Link

Let us start your Data Science Journey together

Acknowledgements

This course is adopted from Data 8, “The Foundations of Data Science” course taught to first-year students at UC Berkeley. All materials for the course, including the textbook and assignments, are available for free online under a Creative Commons license. Textbook: Computational and Inferential Thinking: The Foundations of Data Science is a free online textbook that includes interactive Jupyter notebooks and public data sets for all examples. Course Materials: The Embedded Demo, Lab and Homework Notebooks as well as linked references all come from publicly available materials used in the course during the Spring 2020 semester at UC Berkeley as well as materials used in the self-paced Data 8X course on EdX .

Announcements

• Data Fundamentals classes have started! .

Mid Course Review

Toggle Dark Mode

Data Science / Finance

• Toronto, ON
• Custom Social Profile Link

Spring 2020

• GSI : Hubert Luo
• Email : [first name][last name]@[usual email domain]
• Lab Website : https://hluo27.github.io/data8
• Lab : Friday 12-2pm in Evans 458
• Office Hours : Friday 4-5pm in Barrows 581

Lab Slides (Spring 2020)

Lab 1: Introduction

Lab 2: Causality and Table Operations

Lab 3: Data Types and Table Manipulation

Lab 4: Plots and Functions

Project 1 Lab: Groups, Joins, and Pivots

Lab 5: Iteration and Conditionals

Lab 7: A/B Testing

Project 2 Lab: Bootstrap and Confidence Intervals

Lab 8: Central Limit Theorem, Sample Means, and Correlation

Lab 9: Linear Regression and Residuals

Project 3 Lab: Residuals and Regression Inference

Lab 10: Classification, k-Nearest Neighbours, and Conditional Probability

• Lab : Friday 9-11am in Evans B6
• Office Hours : Friday 2-3pm in Barrows 581

Lab Slides (Fall 2019)

Lab 2: Causality, Expressions, and Table Operations

Lab 3: Data Types, Sequences, and Tables

Lab 5: Probability and Sampling

Lab 6: Hypothesis Testing

Lab 8: Bootstrapping and Confidence Intervals

Lab 9: Central Limit Theorem and Sample Means

Lab 10: Inference Review (Hypothesis Testing, A/B Testing, and Confidence Intervals)

Lab 11: Classification and k-Nearest Neighbours

Lab 12: Residuals

Lab Notebooks (Fall 2019)

Announcements

• Fri 03 December 2021
• Project 3 is due today!
• Pre-final class performance summary: Ed #2919
• RRR week review sessions and OH will be posted on Ed this weekend
• Please fill out the course evaluations !
• Final exam will be in person; please read Ed #2600
• Mon 22 November 2021
• All OHs, discussion, lab and tutoring sections cancelled this week
• Homework 12 due Thursday, 12/2. Submit by Wednesday for 5 bonus points
• Project 3 due Friday, 12/3. Submit by Thursday for 5 bonus points
• There will be a lab in the week after Thanksgiving
• Wed 17 November 2021
• Final exam will be in person
• Homework 11 due Thursday, 11/18. Submit by Wednesday for 5 bonus points
• Project 3 checkpoint due Friday, 11/19. Full project due Friday, 12/3.
• Lab this week is Project 3
• Mon 15 November 2021
• Project 3 has been released! Checkpoint due Friday, 11/19. Full project due Friday, 12/3.
• Homework 12 (the last homework) will be released on Friday, 11/19
• Lab 10 (the last lab) will be released on Monday, 11/29
• Wed 10 November 2021
• Homework 10 due Fri 11/12 (5 bonus points if submitted by 11/11).
• Tomorrow is Veteran's Day holiday. No office hours or labs will be held that day. If your regularly scheduled lab is on a Thursday, your lab GSI will hold an alternate lab time on Wednesday or Friday (before lecture).
• To earn Lab 9 credit: Students in Thursday labs can attend their GSIs alternate lab (no other lab) and get checked off. If you cannot attend the alternate time, complete the lab on your own with all tests passed, by Friday, 11/12, at noon. More details here .
• Fri 05 November 2021
• Extension: Project 2 deadlines have been extended by one day. Project 2 is due tomorrow, 11/6, at 11:59pm. Submit by tonight, 11:59pm, for 5 bonus points.
• Extension: Homework 9 has been extended by one day. Due tonight, 11:59pm.
• Next Thursday 11/11 is Veteran's Day holiday. No labs will be held that day. If your regularly scheduled lab is on a Thursday, your lab GSI will hold an alternate lab time on Wednesday or Friday; details will be posted on Ed by Monday morning. If you cannot attend the alternate time, complete the lab on your own with all tests passed, by Friday, 11/12, at noon.
• Wed 03 November 2021
• HW 9 is due Thu 11/04. Turn it in by tonight for 5 bonus points
• Project 2 due Fri 11/05
• Confidence Intervals Guide: Ed #1967
• Mon 01 November 2021
• HW 9 is due Thu 11/04. Turn it in by Wed 11/03 for 5 bonus points
• Project 2 Party on Tuesday 1-3pm in Moffitt 145 and 7-9pm on Zoom (watch Ed for link)
• Fri 29 October 2021
• Tutoring begins this week; sign up instructions at Ed #1820
• HW 9 will be released today, due 11/4
• Project 2 checkpoint due tonight, project due Fri 11/5
• Mon 25 October 2021
• HW 8 due Thurs 10/28
• Project 2 checkpoint due Fri 10/29, project due Fri 11/5
• Fri 22 October 2021
• Midterm regrade requests are due by tonight: Ed #1772
• Tutoring sign-ups have been released: Ed #1820
• Project 2 will be released today: checkpoint Fri 10/29, due Fri 11/5
• HW 8 will be released today: due Thurs 10/28
• Wed 20 October 2021
• Midterm regrade requests are due by Friday 10/22: Ed #1772
• No homework is due this week
• Project 2 will be released on Friday: checkpoint Fri 10/29, due Fri 11/5
• HW 8 will be released on Friday: due Thurs 10/28
• Mon 18 October 2021
• Mid-semester performance summary has been released! Ed Post #1772.
• Lab 7 will be released today
• Project 2 release on Friday (checkpoint Fri 10/29, due Fri 11/5)
• Fri 15 October 2021
• The midterm is tonight! Relax, you got this!!! :)
• No homework will be released after lecture.
• Regular routine will start back up next week: lab will be released on Monday.
• Wed 13 October 2021
• HW 7 due tomorrow by 11:59pm or turn it in tonight for 5 bonus points!
• HW 7 solutions will be released by 11am on Friday, 10/15.
• Midterm is Friday, 10/15, 7-8:30pm. See Ed #1139 , #1140 , #1310 , #1311 . Scope: Everything through Chapter 12.
• Review sessions this week: Mon 5-7pm, Tue 10am-noon, Tue 5-7pm.
• No homework will be released on Friday, 10/15.
• Mon 11 October 2021
• HW 7 due Thursday, 10/14. Turn in by 10/13 11:59pm for 5 bonus points.
• Wed 06 October 2021
• HW 6 due Thursday, 10/7. Turn in by tonight 11:59pm for 5 bonus points.
• Midterm is Friday, 10/15, 7-8:30pm. See Ed #1139 and #1140 . Scope: Everything through Chapter 12.
• If you did not fill out the Google form but would like to take the midterm at the alternate time of 8:30pm-10pm, email [email protected] by Friday 10/8.
• HW 4 score released, regrades due by tonight.
• Mon 04 October 2021
• HW6 due Thursday, 10/7. Turn in by Wednesday, 10/6 11:59pm for 5 bonus points.
• Midterm is Friday, 10/15, 7-8:30pm. See Edstem #1139 and #1140 . Scope: Everything through Chapter 12.
• Fri 01 October 2021
• Midterm logistics and proctoring details have been posted on Ed. Please read through both posts and ask questions ASAP.
• Project 1 due tonight by 11:59pm PT!
• Mon 27 September 2021
• Project party today Monday, 9/27, 3-5pm in Moffitt 145 and 7-9pm online.
• HW5 due Thursday, 9/30. Turn in by Wednesday, 9/29 11:59pm for 5 bonus points.
• Fri 24 September 2021
• Respond by today about midterm via this form on Ed .
• Project 1 checkpoint due today, 9/24.
• Wed 22 September 2021
• Midterm is Friday, 10/15, 7-8:30pm. Please fill out this form on Ed .
• HW4 due tomorrow, Thursday, 9/23 or tonight by 11:59pm for 5 bonus points
• Project 1 checkpoint due Friday, 9/24. Find a project partner in your lab section!
• Fri 17 September 2021
• HW4 is due 9/23 (turn in by 9/22 for 5 bonus points)
• Project 1 released today. Work on it in lab next week. Checkpoint due 9/24, entire project due 10/1. You can work with with up to one partner from your own lab section.
• Information midterm logistics will be released later today. We will also release a mandatory form on midterm attendance.
• Wed 15 September 2021
• HW 3 is due Thursday, 9/16. Submit by Wednesday for 5 bonus points.
• Starting Wednesday 9/15 there will be a new link for lectures
• The alternate final exam will be Monday, 12/13, 11am-2pm. See here if this is an issue for you.
• Mon 13 September 2021
• Midterm information will be posted on Ed by the end of this week.
• Starting Wednesday 9/15 there will be a new link for lectures. Link will be posted on Ed today.
• Starting next week, you will have to log in with your berkeley.edu account to access the OH queue. Instructions will be posted on Ed.
• Fri 10 September 2021
• HW 3 is being released today and due Thursday, 9/16. Submit by Wednesday for 5 bonus points.
• HW 1 and Lab 2 solutions have been released. Check Ed post #193 for how to view solutions.
• Wed 08 September 2021
• HW 2 is due Thursday, 9/9. Submit by tonight for 5 bonus points.
• Regrade requests for Lab 1 are due tonight!
• Instructor office hours: Prof. Adhikari: after lecture, Memorial Glade. Prof. Wagner: Tuesdays 11am-noon, Bechtel Terrace.
• Fri 03 September 2021
• Sign up for tutoring sections!
• Lab 1 score released later today. Regrades due by next Wednesday, 9/8.
• HW 2 is due Thursday, 9/9. Submit Wednesday, 9/8 for 5 bonus points.
• Wed 01 September 2021
• Sharing code or pictures of code with others is not ok -- not even "what am I doing wrong?". Ask these questions in a private question on Ed or in office hours.
• Tutoring section sign ups are out! Sign up here .
• If you got off the waitlist and need a lab section assignment, follow the directions in this Ed post .
• Mon 30 August 2021
• Homework 1 is due this Thursday, 9/2 at 11:59pm or Wednesday for 5 bonus points.
• OH began today! You may find OH times and locations here .
• Check out our staff-created Data 8 Resources Walkthrough Videos to learn how to navigate the website, Datahub, Zoom, etc.
• Official small group tutoring will be released tomorrow on Ed .
• Take care of yourselves!!!
• Fri 27 August 2021
• Homework 1 is being released today! Due next Thursday, 9/2 at 11:59pm or next Wednesday for 5 bonus points.
• Official small group tutoring will be released sometime next week on Ed . Keep an eye out!
• Take care of yourselves! Stay hydrated and get lots of rest!!
• Wed 18 August 2021
• Welcome to Data 8! You can find our tentative course schedule below.
• Check that you are enrolled in Data 8's Ed page ! Check Ed daily for all course announcements.
• Please sign up for your lab section preferences here by Friday, 8/20 at 11:59pm PT.

Instructors : David Wagner and Ani Adhikari

Lecture : MWF 10am-11am

• Equity & Inclusion

Data 8: Foundations of Data Science

Foundations of Data Science: A Data Science Course for Everyone

What is it?

Foundations of Data Science (Data C8, also listed as COMPSCI/STAT/INFO C8) is a course that gives you a new lens through which to explore the issues and problems that you care about in the world. You will learn the core concepts of inference and computing, while working hands-on with real data including economic data, geographic data and social networks.

Who is it for?

The course is designed for entry-level students from any major. It is designed specifically for students who have not previously taken statistics or computer science courses.

Does it satisfy requirements?

Yes, it satisfies the L&S Quantitative Reasoning requirement, as well as the statistics requirement in most departments that require it. But unlike other statistics courses, it also introduces you to computer programming. Unlike other computer science courses, it gives you a chance to work hands-on with real data and tackle real-world issues.

What are the prerequisites?

Why should i take this course.

In a world in which we’re surrounded by data, this course enables you to combine that data with Python programming skills to ask questions and explore problems that you encounter in any field of study, in a future job, and even in everyday life.

What kind of support is available if I need help?

Lots of support and tutoring are available. In weekly labs, you’ll get hands-on practice. Small-group tutoring sessions are also available, and course staff strongly encourage you to take advantage of their office hours. The Data Scholars program provides community and support to underrepresented and nontraditional students.

What is the format?

The course includes lectures, a required weekly lab, homework, and projects in which you’ll tackle real-life issues using real data. Connector courses connect Data 8 to many areas, if you want to take one at the same time or later.

What does it cover?

The course teaches you all the key ideas of an introductory statistics class, in a new, modern, hands-on way. It weaves in contextual issues like data privacy and bias. At the same time, it gives you a powerful understanding of key ideas in computing.

Data 8 is a great class if you want to understand data in the world around us, or use these tools in your own major. It's also the best foundation for going on in data science.

What do students say about it?

"It's very hands-on -- I was able to go through the data myself, and ask questions I was curious about. It really cemented the idea for me that data tells a story and can shed light on things that I never considered before.”

“Taking data science really sets you apart and makes you more valuable. It lets you ask questions and find answers in a way that most other students can’t. Data helps you communicate, no matter whether your field is technical or non-technical.”

“One of the things I most enjoy about data science is the diversity -- my classmates range from English majors to bio majors to fellow computer science majors -- all looking at data from our different perspectives.”

“I truly came out of this class loving statistics, and I intend to explore more of the world of data science. As someone who is majoring in the social sciences (Political Science) I'm grateful to you and the staff for making this course accessible to all majors across campus.”

“I enjoy that you learn by doing -- you look at the data yourself, analyze it, ask questions, and draw conclusions.”

How can I get more information?

All the course materials are online. Just visit data8.org .

The course has 8 homework sets plus a Final , according to the schedule below . The Final carries twice the weight of a homework.

• The questions are multiple-choice. This doesn't mean simplistic; some questions necessitate running a full experiment.
• Some questions are theoretical (mathematical, conceptual), and others are experimental (programming, data). The questions vary from easy to hard.
• Familiarity with some programming language or platform will help in the homework, e.g., packages for basic matrix operations and quadratic programming are needed.
• The discussion forum has numerous threads about homework and Final questions.

Homework Schedule

Take the course at your own pace. A 'week' below can be spread out to a longer period, e.g., 10 days or two weeks, to fit your schedule.

• Week 1: Do Homework 1 after watching Lectures 1 and 2. Check Solution key 1 after you finish the homework.
• Week 2: Do Homework 2 after watching Lectures 3 and 4. Check Solution key 2 after you finish the homework.
• Week 3: Do Homework 3 after watching Lectures 5 and 6. Check Solution key 3 after you finish the homework.
• Week 4: Do Homework 4 after watching Lectures 7 and 8. Check Solution key 4 after you finish the homework.
• Week 5: Do Homework 5 after watching Lectures 9 and 10. Check Solution key 5 after you finish the homework.
• Week 6: Do Homework 6 after watching Lectures 11 and 12. Check Solution key 6 after you finish the homework.
• Week 7: Do Homework 7 after watching Lectures 13 and 14. Check Solution key 7 after you finish the homework.
• Week 8: Do Homework 8 after watching Lectures 15 and 16. Check Solution key 8 after you finish the homework.
• Week 9: Start on the Final after watching Lectures 17 and 18. Consult the Machine Learning Video Library as needed.
• Week 10: Finish the Final then check the Final solution key . Congratulations! You have completed the course.
• The lectures
• Video Segments
• The instructor

Machine Learning DS-GA 1003 · Spring 2023 · NYU Center for Data Science

About this course.

This course covers a wide variety of introductory topics in machine learning and statistical modeling, including statistical learning theory, convex optimization, generative and discriminative models, kernel methods, boosting, latent variable models and so on. The primary goal is to provide students with the tools and principles needed to solve the data science problems found in practice. This course was designed as part of the core curriculum for the Center for Data Science's Masters degree in Data Science , and is intended as a continuation of DS-GA-1001 Intro to Data Science. This course also serves as a foundation on which more specialized courses and further independent study can build. Course syllabus can be found here .

For registration information, please contact Tina Lam .

Prerequisites

If you'd like to waive the prerequisites, please send an email to Mengye Ren ([email protected]), Ravid Shwartz-Ziv ([email protected]). Note that this course requires some basic understanding of machine learning (covered by DS-GA-1001). For each prerequisite, please clearly list which courses you've taken are equivalent, and highlight it in the transcript. In addition, please complete the Prerequisite Questionnaire for self-assessment.

• DS-GA-1001: Intro to Data Science or its equivalent
• DS-GA-1002: Statistical and Mathematical Methods or its equivalent
• Solid mathematical background , equivalent to a 1-semester undergraduate course in each of the following: linear algebra, multivariate calculus (primarily differential calculus), probability theory, and statistics. (The coverage in the 2015 version of DS-GA 1002, linked above, is sufficient.)
• Python programming required for most homework assignments.
• Recommended: Computer science background up to a "data structures and algorithms" course
• Recommended: At least one advanced, proof-based mathematics course
• Some prerequisites may be waived with permission of the instructor
• You can self-assess your preparation by filling out the Prerequisite Questionnaire
• Lecture format : In-person with Zoom recording
• Lecture location : GCASL_C95 - Global Center for Academic & Spiritual Life, 238 Thompson Street, New York, NY, 10012-1020
• Office hours :
• Colin Wan: Mon 5:00PM - 6:00PM; Room 204 (60 5th Ave)
• Ying Wang: Wed 6:00PM - 7:00PM; Room 204 (60 5th Ave)
• Yanlai Yang: Wed 1:00PM - 2:00PM; Room 242 (60 5th Ave)
• Discussions : We will use Campuswire for class discussion. Rather than emailing questions to the teaching staff, please post your questions on Campuswire, where they will be answered by the instructors, TAs, graders, and other students. For questions that are not specific to the class, you are also encouraged to post to Stack Overflow for programming questions and Cross Validated for statistics and machine learning questions. Please also post a link to these postings in Campuswire, so others in the class can answer the questions and benefit from the answers.
• Homework (40%). There are 7-8 homeworks. See Assignments section for details. Some homeworks may have optional problems. You should view the optional problems primarily as a way to engage with more material, if you have the time. They will be counted towards extra credit.
• Exams: midterm (30%) + final (30%). Midterm: Mar 14, 2023.
• Midterm exam Midterm exam solution
• Final exam Final exam solution
• Extra credits (2%). You will be awarded with up to 2% extra credit if you answer other students' questions in a substantial and helpful way on Campuswire. Extra credits may bump up your grade up to half a grade (e.g. B to B+).

Related courses

• Spring 2022 offering of DS-GA-1003
• Foundations of Maching Learning from Bloomberg ML EDU by David S. Rosenberg (with videos)

Other tutorials and references

Assignments.

Late Policy: Homeworks are due at 11:59 PM EST on the date specified. You have seven late days in total which can be used throughout the semester without penalty. Once you run out of late days, each additional late day will incur a 20% penalty . For example, if you submit an assignment 1 day late after using all your late days, a score of 90 will only be counted as 72. Note that the maximum late days per homework is two days , meaning that Gradescope will not accept submissions 48 hours after the due date.

Collaboration Policy: You may form study groups and discuss problems with your classmates. However, you must write up the homework solutions and the code from scratch, without referring to notes from your joint session. In your solution to each problem, you must write down the names of any person with whom you discussed the problem—this will not affect your grade.

Submission: Homework should be submitted through Gradescope . If you have not used Gradescope before, please watch this short video: "For students: submitting homework." At the beginning of the semester, you will be added to the Gradescope class roster. This will give you access to the course page, and the assignment submission form. To submit assignments, you will need to:

• Upload a single PDF document containing all the math, code, plots, and exposition required for each problem.
• Where homework assignments are divided into sections, please begin each section on a new page .
• You will then select the appropriate page ranges for each homework problem, as described in the "submitting homework" video.

Feedback: Check Gradescope to get your scores on each individual problem, as well as comments on your answers. Regrading requests should be submitted on Gradescope.

Typesetting your homework

Due: January 1st, 11:59 PM EST

Error Decomposition and Polynomial Regression

Due: February 1st, 11:59 PM EST

Gradient Descent & Regularization

Due: February 15th, 11:59 PM EST

SVMs and Kernel Methods

Due: March 1st, 11:59 PM EST

Probabilistic models

Due: March 22nd, 11:59 PM EST

Multiclass Linear SVM

Due: April 5th, 11:59 PM EST

Decision Trees and Boosting

Due: April 19th, 11:59 PM EST

Computation Graphs, Back-propagation, and Neural Networks

Due: May 3rd, 11:59 PM EST

Instructors

[email protected]

Mengye Ren is an Assistant Professor of Computer Science and Data Science at NYU. His research focuses on deep learning and computer vision.

Ravid Shwartz-Ziv

[email protected]

Ravid Shwartz-Ziv is a Faculty Fellow at NYU CDS. His research focuses on machine learning.

Section Leaders

[email protected]

Colin is an NYU alumnus. He graduated from the CDS Master's program in 2022.

[email protected]

Ying is a second-year Masters student in the Data Science program at NYU CDS.

Yanlai Yang

[email protected]

Yanlai is a first-year PhD student in computer science at NYU Courant.

Xiaojing Fan

[email protected]

Xiaojing is a second-year Masters student in the Data Science program at NYU CDS.

[email protected]

Junze is a second-year Masters student in the Data Science program at NYU CDS

Richard-John Lin

[email protected]

Richard is a second-year Masters student in the Data Science program at NYU CDS.

[email protected]

Jerry Xue is a second-year Master student in the Data Science program at NYU CDS.

Frances Yuan

[email protected]

Frances is a second-year Masters student in the Data Science program at NYU CDS.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

ECE340_HW8_FA15_solution.pdf

Latest commit, file metadata and controls.