data 8 homework solutions github

Data Science / Finance

• Toronto, ON
• Custom Social Profile Link

Spring 2020

• GSI : Hubert Luo
• Email : [first name][last name]@[usual email domain]
• Lab Website : https://hluo27.github.io/data8
• Lab : Friday 12-2pm in Evans 458
• Office Hours : Friday 4-5pm in Barrows 581

Lab Slides (Spring 2020)

Lab 1: Introduction

Lab 2: Causality and Table Operations

Lab 3: Data Types and Table Manipulation

Lab 4: Plots and Functions

Project 1 Lab: Groups, Joins, and Pivots

Lab 5: Iteration and Conditionals

Lab 7: A/B Testing

Project 2 Lab: Bootstrap and Confidence Intervals

Lab 8: Central Limit Theorem, Sample Means, and Correlation

Lab 9: Linear Regression and Residuals

Project 3 Lab: Residuals and Regression Inference

Lab 10: Classification, k-Nearest Neighbours, and Conditional Probability

• Lab : Friday 9-11am in Evans B6
• Office Hours : Friday 2-3pm in Barrows 581

Lab Slides (Fall 2019)

Lab 2: Causality, Expressions, and Table Operations

Lab 3: Data Types, Sequences, and Tables

Lab 5: Probability and Sampling

Lab 6: Hypothesis Testing

Lab 8: Bootstrapping and Confidence Intervals

Lab 9: Central Limit Theorem and Sample Means

Lab 10: Inference Review (Hypothesis Testing, A/B Testing, and Confidence Intervals)

Lab 11: Classification and k-Nearest Neighbours

Lab 12: Residuals

Lab Notebooks (Fall 2019)

Announcements

• Fri 03 December 2021
• Project 3 is due today!
• Pre-final class performance summary: Ed #2919
• RRR week review sessions and OH will be posted on Ed this weekend
• Please fill out the course evaluations !
• Final exam will be in person; please read Ed #2600
• Mon 22 November 2021
• All OHs, discussion, lab and tutoring sections cancelled this week
• Homework 12 due Thursday, 12/2. Submit by Wednesday for 5 bonus points
• Project 3 due Friday, 12/3. Submit by Thursday for 5 bonus points
• There will be a lab in the week after Thanksgiving
• Wed 17 November 2021
• Final exam will be in person
• Homework 11 due Thursday, 11/18. Submit by Wednesday for 5 bonus points
• Project 3 checkpoint due Friday, 11/19. Full project due Friday, 12/3.
• Lab this week is Project 3
• Mon 15 November 2021
• Project 3 has been released! Checkpoint due Friday, 11/19. Full project due Friday, 12/3.
• Homework 12 (the last homework) will be released on Friday, 11/19
• Lab 10 (the last lab) will be released on Monday, 11/29
• Wed 10 November 2021
• Homework 10 due Fri 11/12 (5 bonus points if submitted by 11/11).
• Tomorrow is Veteran's Day holiday. No office hours or labs will be held that day. If your regularly scheduled lab is on a Thursday, your lab GSI will hold an alternate lab time on Wednesday or Friday (before lecture).
• To earn Lab 9 credit: Students in Thursday labs can attend their GSIs alternate lab (no other lab) and get checked off. If you cannot attend the alternate time, complete the lab on your own with all tests passed, by Friday, 11/12, at noon. More details here .
• Fri 05 November 2021
• Extension: Project 2 deadlines have been extended by one day. Project 2 is due tomorrow, 11/6, at 11:59pm. Submit by tonight, 11:59pm, for 5 bonus points.
• Extension: Homework 9 has been extended by one day. Due tonight, 11:59pm.
• Next Thursday 11/11 is Veteran's Day holiday. No labs will be held that day. If your regularly scheduled lab is on a Thursday, your lab GSI will hold an alternate lab time on Wednesday or Friday; details will be posted on Ed by Monday morning. If you cannot attend the alternate time, complete the lab on your own with all tests passed, by Friday, 11/12, at noon.
• Wed 03 November 2021
• HW 9 is due Thu 11/04. Turn it in by tonight for 5 bonus points
• Project 2 due Fri 11/05
• Confidence Intervals Guide: Ed #1967
• Mon 01 November 2021
• HW 9 is due Thu 11/04. Turn it in by Wed 11/03 for 5 bonus points
• Project 2 Party on Tuesday 1-3pm in Moffitt 145 and 7-9pm on Zoom (watch Ed for link)
• Fri 29 October 2021
• Tutoring begins this week; sign up instructions at Ed #1820
• HW 9 will be released today, due 11/4
• Project 2 checkpoint due tonight, project due Fri 11/5
• Mon 25 October 2021
• HW 8 due Thurs 10/28
• Project 2 checkpoint due Fri 10/29, project due Fri 11/5
• Fri 22 October 2021
• Midterm regrade requests are due by tonight: Ed #1772
• Tutoring sign-ups have been released: Ed #1820
• Project 2 will be released today: checkpoint Fri 10/29, due Fri 11/5
• HW 8 will be released today: due Thurs 10/28
• Wed 20 October 2021
• Midterm regrade requests are due by Friday 10/22: Ed #1772
• No homework is due this week
• Project 2 will be released on Friday: checkpoint Fri 10/29, due Fri 11/5
• HW 8 will be released on Friday: due Thurs 10/28
• Mon 18 October 2021
• Mid-semester performance summary has been released! Ed Post #1772.
• Lab 7 will be released today
• Project 2 release on Friday (checkpoint Fri 10/29, due Fri 11/5)
• Fri 15 October 2021
• The midterm is tonight! Relax, you got this!!! :)
• No homework will be released after lecture.
• Regular routine will start back up next week: lab will be released on Monday.
• Wed 13 October 2021
• HW 7 due tomorrow by 11:59pm or turn it in tonight for 5 bonus points!
• HW 7 solutions will be released by 11am on Friday, 10/15.
• Midterm is Friday, 10/15, 7-8:30pm. See Ed #1139 , #1140 , #1310 , #1311 . Scope: Everything through Chapter 12.
• Review sessions this week: Mon 5-7pm, Tue 10am-noon, Tue 5-7pm.
• No homework will be released on Friday, 10/15.
• Mon 11 October 2021
• HW 7 due Thursday, 10/14. Turn in by 10/13 11:59pm for 5 bonus points.
• Wed 06 October 2021
• HW 6 due Thursday, 10/7. Turn in by tonight 11:59pm for 5 bonus points.
• Midterm is Friday, 10/15, 7-8:30pm. See Ed #1139 and #1140 . Scope: Everything through Chapter 12.
• If you did not fill out the Google form but would like to take the midterm at the alternate time of 8:30pm-10pm, email [email protected] by Friday 10/8.
• HW 4 score released, regrades due by tonight.
• Mon 04 October 2021
• HW6 due Thursday, 10/7. Turn in by Wednesday, 10/6 11:59pm for 5 bonus points.
• Midterm is Friday, 10/15, 7-8:30pm. See Edstem #1139 and #1140 . Scope: Everything through Chapter 12.
• Fri 01 October 2021
• Midterm logistics and proctoring details have been posted on Ed. Please read through both posts and ask questions ASAP.
• Project 1 due tonight by 11:59pm PT!
• Mon 27 September 2021
• Project party today Monday, 9/27, 3-5pm in Moffitt 145 and 7-9pm online.
• HW5 due Thursday, 9/30. Turn in by Wednesday, 9/29 11:59pm for 5 bonus points.
• Fri 24 September 2021
• Respond by today about midterm via this form on Ed .
• Project 1 checkpoint due today, 9/24.
• Wed 22 September 2021
• Midterm is Friday, 10/15, 7-8:30pm. Please fill out this form on Ed .
• HW4 due tomorrow, Thursday, 9/23 or tonight by 11:59pm for 5 bonus points
• Project 1 checkpoint due Friday, 9/24. Find a project partner in your lab section!
• Fri 17 September 2021
• HW4 is due 9/23 (turn in by 9/22 for 5 bonus points)
• Project 1 released today. Work on it in lab next week. Checkpoint due 9/24, entire project due 10/1. You can work with with up to one partner from your own lab section.
• Information midterm logistics will be released later today. We will also release a mandatory form on midterm attendance.
• Wed 15 September 2021
• HW 3 is due Thursday, 9/16. Submit by Wednesday for 5 bonus points.
• Starting Wednesday 9/15 there will be a new link for lectures
• The alternate final exam will be Monday, 12/13, 11am-2pm. See here if this is an issue for you.
• Mon 13 September 2021
• Midterm information will be posted on Ed by the end of this week.
• Starting Wednesday 9/15 there will be a new link for lectures. Link will be posted on Ed today.
• Starting next week, you will have to log in with your berkeley.edu account to access the OH queue. Instructions will be posted on Ed.
• Fri 10 September 2021
• HW 3 is being released today and due Thursday, 9/16. Submit by Wednesday for 5 bonus points.
• HW 1 and Lab 2 solutions have been released. Check Ed post #193 for how to view solutions.
• Wed 08 September 2021
• HW 2 is due Thursday, 9/9. Submit by tonight for 5 bonus points.
• Regrade requests for Lab 1 are due tonight!
• Instructor office hours: Prof. Adhikari: after lecture, Memorial Glade. Prof. Wagner: Tuesdays 11am-noon, Bechtel Terrace.
• Fri 03 September 2021
• Sign up for tutoring sections!
• Lab 1 score released later today. Regrades due by next Wednesday, 9/8.
• HW 2 is due Thursday, 9/9. Submit Wednesday, 9/8 for 5 bonus points.
• Wed 01 September 2021
• Sharing code or pictures of code with others is not ok -- not even "what am I doing wrong?". Ask these questions in a private question on Ed or in office hours.
• Tutoring section sign ups are out! Sign up here .
• If you got off the waitlist and need a lab section assignment, follow the directions in this Ed post .
• Mon 30 August 2021
• Homework 1 is due this Thursday, 9/2 at 11:59pm or Wednesday for 5 bonus points.
• OH began today! You may find OH times and locations here .
• Check out our staff-created Data 8 Resources Walkthrough Videos to learn how to navigate the website, Datahub, Zoom, etc.
• Official small group tutoring will be released tomorrow on Ed .
• Take care of yourselves!!!
• Fri 27 August 2021
• Homework 1 is being released today! Due next Thursday, 9/2 at 11:59pm or next Wednesday for 5 bonus points.
• Official small group tutoring will be released sometime next week on Ed . Keep an eye out!
• Take care of yourselves! Stay hydrated and get lots of rest!!
• Wed 18 August 2021
• Welcome to Data 8! You can find our tentative course schedule below.
• Check that you are enrolled in Data 8's Ed page ! Check Ed daily for all course announcements.
• Please sign up for your lab section preferences here by Friday, 8/20 at 11:59pm PT.

Instructors : David Wagner and Ani Adhikari

Lecture : MWF 10am-11am

Machine Learning DS-GA 1003 · Spring 2023 · NYU Center for Data Science

About this course.

This course covers a wide variety of introductory topics in machine learning and statistical modeling, including statistical learning theory, convex optimization, generative and discriminative models, kernel methods, boosting, latent variable models and so on. The primary goal is to provide students with the tools and principles needed to solve the data science problems found in practice. This course was designed as part of the core curriculum for the Center for Data Science's Masters degree in Data Science , and is intended as a continuation of DS-GA-1001 Intro to Data Science. This course also serves as a foundation on which more specialized courses and further independent study can build. Course syllabus can be found here .

For registration information, please contact Tina Lam .

Prerequisites

If you'd like to waive the prerequisites, please send an email to Mengye Ren ([email protected]), Ravid Shwartz-Ziv ([email protected]). Note that this course requires some basic understanding of machine learning (covered by DS-GA-1001). For each prerequisite, please clearly list which courses you've taken are equivalent, and highlight it in the transcript. In addition, please complete the Prerequisite Questionnaire for self-assessment.

• DS-GA-1001: Intro to Data Science or its equivalent
• DS-GA-1002: Statistical and Mathematical Methods or its equivalent
• Solid mathematical background , equivalent to a 1-semester undergraduate course in each of the following: linear algebra, multivariate calculus (primarily differential calculus), probability theory, and statistics. (The coverage in the 2015 version of DS-GA 1002, linked above, is sufficient.)
• Python programming required for most homework assignments.
• Recommended: Computer science background up to a "data structures and algorithms" course
• Recommended: At least one advanced, proof-based mathematics course
• Some prerequisites may be waived with permission of the instructor
• You can self-assess your preparation by filling out the Prerequisite Questionnaire
• Lecture format : In-person with Zoom recording
• Lecture location : GCASL_C95 - Global Center for Academic & Spiritual Life, 238 Thompson Street, New York, NY, 10012-1020
• Office hours :
• Colin Wan: Mon 5:00PM - 6:00PM; Room 204 (60 5th Ave)
• Ying Wang: Wed 6:00PM - 7:00PM; Room 204 (60 5th Ave)
• Yanlai Yang: Wed 1:00PM - 2:00PM; Room 242 (60 5th Ave)
• Discussions : We will use Campuswire for class discussion. Rather than emailing questions to the teaching staff, please post your questions on Campuswire, where they will be answered by the instructors, TAs, graders, and other students. For questions that are not specific to the class, you are also encouraged to post to Stack Overflow for programming questions and Cross Validated for statistics and machine learning questions. Please also post a link to these postings in Campuswire, so others in the class can answer the questions and benefit from the answers.
• Homework (40%). There are 7-8 homeworks. See Assignments section for details. Some homeworks may have optional problems. You should view the optional problems primarily as a way to engage with more material, if you have the time. They will be counted towards extra credit.
• Exams: midterm (30%) + final (30%). Midterm: Mar 14, 2023.
• Midterm exam Midterm exam solution
• Final exam Final exam solution
• Extra credits (2%). You will be awarded with up to 2% extra credit if you answer other students' questions in a substantial and helpful way on Campuswire. Extra credits may bump up your grade up to half a grade (e.g. B to B+).

Related courses

• Spring 2022 offering of DS-GA-1003
• Foundations of Maching Learning from Bloomberg ML EDU by David S. Rosenberg (with videos)

Other tutorials and references

Assignments.

Late Policy: Homeworks are due at 11:59 PM EST on the date specified. You have seven late days in total which can be used throughout the semester without penalty. Once you run out of late days, each additional late day will incur a 20% penalty . For example, if you submit an assignment 1 day late after using all your late days, a score of 90 will only be counted as 72. Note that the maximum late days per homework is two days , meaning that Gradescope will not accept submissions 48 hours after the due date.

Collaboration Policy: You may form study groups and discuss problems with your classmates. However, you must write up the homework solutions and the code from scratch, without referring to notes from your joint session. In your solution to each problem, you must write down the names of any person with whom you discussed the problem—this will not affect your grade.

Submission: Homework should be submitted through Gradescope . If you have not used Gradescope before, please watch this short video: "For students: submitting homework." At the beginning of the semester, you will be added to the Gradescope class roster. This will give you access to the course page, and the assignment submission form. To submit assignments, you will need to:

• Upload a single PDF document containing all the math, code, plots, and exposition required for each problem.
• Where homework assignments are divided into sections, please begin each section on a new page .
• You will then select the appropriate page ranges for each homework problem, as described in the "submitting homework" video.

Feedback: Check Gradescope to get your scores on each individual problem, as well as comments on your answers. Regrading requests should be submitted on Gradescope.

Typesetting your homework

Due: January 1st, 11:59 PM EST

Error Decomposition and Polynomial Regression

Due: February 1st, 11:59 PM EST

Gradient Descent & Regularization

Due: February 15th, 11:59 PM EST

SVMs and Kernel Methods

Due: March 1st, 11:59 PM EST

Probabilistic models

Due: March 22nd, 11:59 PM EST

Multiclass Linear SVM

Due: April 5th, 11:59 PM EST

Decision Trees and Boosting

Due: April 19th, 11:59 PM EST

Computation Graphs, Back-propagation, and Neural Networks

Due: May 3rd, 11:59 PM EST

Instructors

[email protected]

Mengye Ren is an Assistant Professor of Computer Science and Data Science at NYU. His research focuses on deep learning and computer vision.

Ravid Shwartz-Ziv

[email protected]

Ravid Shwartz-Ziv is a Faculty Fellow at NYU CDS. His research focuses on machine learning.

Section Leaders

[email protected]

Colin is an NYU alumnus. He graduated from the CDS Master's program in 2022.

[email protected]

Ying is a second-year Masters student in the Data Science program at NYU CDS.

Yanlai Yang

[email protected]

Yanlai is a first-year PhD student in computer science at NYU Courant.

Xiaojing Fan

[email protected]

Xiaojing is a second-year Masters student in the Data Science program at NYU CDS.

[email protected]

Junze is a second-year Masters student in the Data Science program at NYU CDS

Richard-John Lin

[email protected]

Richard is a second-year Masters student in the Data Science program at NYU CDS.

[email protected]

Jerry Xue is a second-year Master student in the Data Science program at NYU CDS.

Frances Yuan

[email protected]

Frances is a second-year Masters student in the Data Science program at NYU CDS.

General Homework Information

This homework will be a set of ungraded practice problems.

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

A custom download tool for Moscow buildings dataset

Casyfill/mosplus

Folders and files, repository files navigation.

A custom download tool for Moscow builsings footprint tax lot data, built with CartoDB as fork of Crhis Whong's Plutoplus. View it live at [ http://casyfill.github.io/mosplus/# ]

Building Footprint is a great Moscow Open Data Resource that contains a wealth of information about the city's building footprints, including adress, cadaster zoning, status, registration data, and few more attributes.It contains information for the city's 145,000+ buildings, and includes 19 attributes for each one. That is (so far) a unique open data collection for Russia!

• Brief Description(rus)
• Dataset Passport

Moscow Building Footprint is quite large, awailable through an API only and hard to use. That is why I used blueprints for a great tool from Chris Wong (originally for MapPluto dataset) to to help people get access to smaller chunks of the data quickly and easily for whatever they are working on. All data is version from 25.03.2016 and can be exported as geoJSON , zipped shapefile , and CSV , or can be imported directly to your cartoDB account . Geometries are exported in WGS84 (Latitude and Longitude). For neighborhood (rayon) borders, this dataset from Gis-Lab was used.

If you like this project, let me know by tweeting to @casy_fill. You can easily fork the original code (thanks to Chris, again). This project was built with the CartoDB web mapping platform. HereChris describes his original pipeline. I will try to add my experience as well. Support open Data!

NOTE: data provided as is. There are and there will be few incorrect entries. There are ~ 7700 entries with the wrong geometry provided. for now, they are cleared out and saved as processing_data/problematic.csv .

Getting the data

to get all data at once: download this repository and make use of Makefile: cd data_processing; make data

Process requires python 2.7 and basic libraries: - pandas - geopandas - requests

• fields properties
• join db, drop geoData
• data_export
• add neighborhoods to the map
• create spatial joint and id
• connect to the frontend
• description
• buttons style
• replace script with whget
• migrate Database
• Jupyter Notebook 53.9%
• JavaScript 17.5%
• Python 2.5%
• Makefile 0.1%