- Spring 2023
Syllabus Description:
Description
Welcome to PHYS 417 "Neural Network Methods for Signals in Engineering and Physical Sciences." This is the second time we teach this class. The course materials were co-developed by Prof. Shih-Chieh Hsu from Physics and Prof. Eli Shlizerman from AMath/ECE. The former course number was PHYS 427 in Spring 2022.
We will provide a practical introduction to Neural Networks, and their application in the analysis of signal data common in engineering and physical sciences. We will build computational skills for training neural networks, understanding, and working with modern algorithms. The course will conclude with projects developing NN models to solve data provided by the NSF HDR Institutes A3D3 (https://a3d3.ai/).
Learning Goal
The learning goal is to enable students to use industry-standard tools (git, Python, PyTorch) to build appropriate neural network models and solve data-driven analyses in engineering and physical sciences.
Requirements
Basic Computer Science and Python Programming Skills at an equivalent level as (ASTR300, AMATH 301, EE 241, CSE160, STAT180)
Classroom Format
The format of instruction will be divided between lectures (theoretical concepts) and labs (practical aspects). Students are required to study pre-lecture recording as well as lab videos prior to the Tuesday lecture time.
Logistics
Instructor: Shih-Chieh Hsu (schsu@uw.edu office hours Monday 3pm - 4pm; PAB B213)
Teaching Assistant:
Ali Garabaglu (agarabag@uw.edu, office hours Wednesday 9:30 am - 10:30 am; PAB B205)
Abdelrahman Elabd (aelabd2@uw.ed, office hours Friday 4pm - 5pm; PAB B243)
Lecture (PAB A110): Mon 11:30pm~12:20pm
Lab Sessions (PAB B143):
Tue 1:30pm~4:20pm (A) Abdelrahman Elabd
Wed 1:30pm~4:20pm (B) Ali Garabaglu
Discord discussion https://discord.gg/VEN58V3HJ8
Credit: 3 credits, graded
Activities
To support the learning goal above, the following activities are incorporated into the course. For a detailed schedule, please see the syllabus table below.
(i) Fundamental Concepts (Canvas quiz - Individual)
To ensure that students gain a strong theoretical foundation, there will be a weekly canvas quiz with ~5 questions related to fundamental concepts discussed in the lectures. You will first fill in your initial answers before the lecture session (due 11:59pm Sunday), and your final answers after the lecture (due 11:59pm Monday). This is intended to be an individual activity.
(ii) Hands-on Practice with Neural Networks (Lab report - Individual)
To introduce deep learning tools and their applications, students will work on a weekly lab assignment in the form of a GitHub report (due 11:59pm Thursday next week). For each assignment, students will solve a particular problem by building and training a neural net. You are highly encouraged to experiment with your neural net architecture, as long as you follow the problem constraints. Discussions with other students and usage of on-line tools are encouraged, but the final lab report "MUST" be your own work. After report submissions, each student will grade 2 other students' reports (due 11:59pm Monday next week). Peer reviews help you to spot gaps in the code training, and give you the opportunity to learn from your cohorts.
(iii) Project (Application to data of choice - Team)
As a culminating project, students will form teams of two and apply the techniques studied in class to one of several real-world scientific datasets. The datasets for the project will be provided later in the quarter. Project submission will include a presentation and a short report in Jupyter Notebook.
Grading
Grading is divided between the three activities of the course:
(i) Fundamental Concepts (Canvas quiz - Individual, weekly): 20%
Each quiz is worth 10 pts and will be graded in two phases: your initial answers before the lecture discussion (5pts, graded for completion) and your final answers after attending the lecture discussion (5pts). Your lowest quiz score will be dropped. This activity will start from week 2. Your lowest lecture quiz score will be dropped.
(ii) Hands-on Practice with Neural Networks (Lab report - Individual, weekly): 40%
Each lab report is worth 10 pts and is evaluated on 1) Lab attendance (1pt), 2) Code completeness/meeting the assignment goal (4pts), 3) Code organization (2pts), 4) Code documentation (2pts) and 5) Peer review (1pt). Your lowest lab report score will be dropped.
(iii) Project (Application to data of choice - Team): 40%
Evaluated based on project presentation and report. More information will follow during the quarter.
Course Policies
Missing a lecture session
You won’t be penalized for missing a lecture, but you will lose an opportunity to discuss the lecture quiz answers and lab reviews with the instruction team.
Late quiz submission
Initial answers - Any quiz question that is left unanswered, makes no sense or doesn't answer the question being asked is considered incomplete and won’t receive a completion score.
Final answers - Only your final answers before the due date will be considered for grading. Modification to your answers after the due date will not be accepted.
Lab report guidelines
For tips regarding forming your lab report, see the lab report guideline.
Missing a lab session
Missing a lab session without approval from the instruction team will result in a deduction of lab participation scores. This may be waived if a student has special circumstances (e.g. feeling sick, personal/family emergencies, etc). If you believe such circumstances apply to you, you must contact the instruction team before the lab session.
Late lab report submission
We will deduct -2pts from the lab report score for each day late.
Schedule
Wk |
Subject |
Lecture discussions |
Lab sessions |
Report |
1 |
Introduction to Neural Networks Methods, Feed Forward Networks |
Intro to Neural Networks |
Python review, Numpy, Matplotlib |
|
2 |
Machine Learning Practices and Problems |
|||
3 |
Deep Learning Practices |
Training Fully Connected Neural Networks (FCN) for MNIST classification |
||
4 |
Convolutional Neural Networks |
Training CNN for Fashion MNIST |
||
5 |
Sequence Models |
Recurrent Neural Networks |
Vanilla RNN, Natural Language Processing example (CharRNN) |
|
6 |
Data Analysis and Modeling of Sequences |
Encoder/Decoder on temporal data |
||
7 |
Data Analysis and Modeling of Sequences with Neural Networks |
Transformer on Natural Language Processing |
||
8 |
Applications to Signals Data |
|
Selecting and understanding the dataset for the project (Analysis, visualization, physical meanings, etc) |
|
9 |
|
Selected project |
||
10 |
Student Poster |
Memorial day (5/29) June 1st 1pm~3pm |
WRF Data Science Studio, 6th floor PAT |
Textbooks and Lecture Notes
Python Programming
Deep Learning
- Deep learning (Links to an external site.) by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (theory and concepts; available online)
- Neural Networks and Deep Learning (Links to an external site.) by Michael Nielsen (concepts and examples; online book)
- Deep Learning with Python by Francois Chollet (learning through examples; Keras)
Practical Deep Learning
- Machine Learning Yearning (Links to an external site.) by Andrew Ng (practical concepts; available online)
- Hands-On Machine Learning with Scikit-Learn and TensorFlow by Aurélien Géron (ML and Tensorflow)
- TensorFlow Deep Learning Cookbook by Antonio Gulli and Amita Kapoor (cookbook examples)
Machine Learning
- A Course in Machine Learning (Links to an external site.) by Hal Daume III (Introduction; available online)
- Machine Learning: A Probabilistic Perspective by Kevin Murphy (Extensive text)
Accessibility
We endeavor to make the course welcoming and accessible to all students. Standard accessibility requests will be handled through DRS.
Accommodations for students with disabilities: In compliance with the University of Washington policy and equal access laws, the course Instructor is available to discuss appropriate academic accommodations that may be required for students with disabilities. Requests for academic accommodations are to be made during the first three weeks of the quarter, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.
Religious Accommodations: Washington state law requires that UW develop a policy for the accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy (https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/). Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form (https://registrar.washington.edu/students/religious-accommodations-request/ (Links to an external site).
Other Policies:
COVID-19 Health and Safety:
EH&S has made a number of changes to the COVID guidelines to bring it up to date with the CDC recommendations. If you have any questions, please take some time to review the information found in the following links.
- COVID-19 Prevention and Response: https://ehs.washington.edu/covid-19-prevention-and-response/covid-19-health-and-safety
- Updated Flowchart: https://ehs.washington.edu/resource/covid-19-public-health-flowchart-updated-21423-1175
Inclusivity Statement: We understand that our members represent a rich variety of backgrounds and perspectives. The University of Washington is committed to providing an atmosphere for learning that respects diversity. While working together to build this community we ask all members to:
- share their unique experiences, values, and beliefs
- be open to the views of others
- honor the uniqueness of their colleagues
- appreciate the opportunity that we have to learn from each other in this community
- value each other’s opinions and communicate in a respectful manner
- keep confidential discussions that the community has of a personal (or professional) nature
- use this opportunity together to discuss ways in which we can create an inclusive environment in this course and across the UW community
Academic Misconduct:
It is essential that students in fulfillment of their academic requirements and in preparation to enter their profession shall adhere to the University of Washington’s Student Code of Conduct. Any student in this course suspected of academic misconduct (e.g., cheating, plagiarism, or falsification) will be reported to the University’s Office of Community Standards and Student conduct.
Student code of conduct applies to all mediums in which course activities are held (virtual lectures, discussion meetings, QA sessions, office hours, virtual discussions). In particular, the virtual discussion board is intended for questions and discussions exclusively related to course material. Anonymous posting is not supported and any offensive language toward students or course staff violates the student code of conduct and is to be reported to the Student Code of Conduct council. Communication is for constructive discussion of material and is not a replacement for QA sessions, lectures, and office hours. In particular, the discussion board is NOT intended for asking about problems in your code, Scheduling appointments, Sharing suggestions and impressions of course proceedings. For sharing feedback, we set up an anonymous feedback box and will appreciate any constructive feedback.