Instructor: Michelle R. Greene, Ph.D

Email: mgreene2@bates.edu

Office hours: Option 1: Book me via calendar
Option 2: Stop by Hathorn 106 - I’m happy to meet when my door is open!

Logistics:

Class: Tuesday and Thursday 9:30 - 10:50 Hathorn 100

Introduction

”The road to true artificial intelligence is not paved with a single discipline, but rather a collection of specialized subject matters, techniques and theories that together interact to create some form of intelligence.” - Christoffer O. Hernoes

Even if you don’t realize it, you use artificial neural networks every day. If you have ever asked a question of Siri, taken a word suggestion from predictive text, or used a whimsical photo filter, you have engaged with a neural network! By examining patterns in vast amounts of data, neural networks are learning many useful tasks from language translation to medical diagnosis. In some cases, neural networks can learn a task so well that they outperform humans – a neural network-based system named Watson won Jeopardy, and a neural network called Alpha Go beat world champion Lee Sedol.

As neural networks deploy in more areas of life, it will be critical to think about how these artificial thinking systems interact with our own intelligence. Already, ethical questions are being raised about the difficulty of understanding the decisions made by neural networks, and some even criticize them as “black boxes”.

The primary goal of this course is to “unbox” neural networks. You will see that these models are actually built from some fairly simple parts, and that their complexity emerges at scale. By implementing these models yourself, you will see first-hand what they do well, and where they struggle. Throughout the course, we will compare the performance of neural networks to the performance of human brains and asking the question: are neural networks really neural?

Learning Objectives:

By the end of the course, you should be able to:

  • Be able to describe the computation being done by a McCulloch-Pitts neuron and implement it in Python.
  • Be able to explain the importance of an activation function, name several common activation functions, and choose an appropriate activation function for a given task.
  • Be able to describe backpropagation at a conceptual level and compare and contrast it with Hebbian learning.
  • Be able to list several tasks that neural networks do well, as well as several that they do not yet do well.
  • Explain the importance of training data for both supervised and unsupervised neural networks, and how various undesirable biases can be perpetuated and amplified by the training data.

Materials:

  • You will need access to an internet-enabled computer for this course. If you do not have your own computer, please contact your Student Support Advisor — they will put you in contact with ILS for a loaner laptop. You may also use any of the computer labs on campus.

  • Readings will be available to you here and on Lyceum.

Grading:

Grading philosophy

No single assignment listed below is worth more than 10% of your final grade. This is intentional. This allows anyone to bounce back from a less-than-optimal score on any assignment. If you approach this course with consistent effort, you will succeed!  

Assessed components

Daily reading annotations in Hypothes.is: 1.25% per reading * 24 readings = 30% of total
We will be using the social reading annotation tool Hypothes.is. This is similar to Perusall, but does not assign you a grade via artificial intelligence. In each reading, you may highlight passages and figures for comment or question, follow up on the comments and questions or your colleagues, and use “upvoting” to endorse questions and comments. This has several advantages over traditional reading:

  • It provides you with a richer experience because each reading becomes a conversation.
  • It allows you to get your questions answered by me or a colleague before class.
  • It allows me to get a sense of what the class is thinking ahead of time.

The expectation for this portion of your grade is meaningful contribution to each session’s readings. Your reading annotations are due at midnight on the night before each class meeting. Here is a quick video tutorial to get you started. Please note: although the video states that they will be using public annotations, we will be using the private group linked above instead.

Daily class notebook: 1.25% per notebook * 24 sessions = 30% of total Each class will feature a Google Colab notebook. This notebook will contain the in-class Python exercises and your class notes. Exercises that are not finished in class should be finished as homework. Each notebook will be due at midnight on the day class was held. Notebooks will be graded in a binary fashion (complete / incomplete).
 

Tutorials: 8% per tutorial * 5 tutorials = 40% of total Throughout the semester, you will create five expanded blog posts in colab notebooks that will explain a particular neural network to a beginner. These notebooks will allow you to reflect upon the functions of the network, their historical context, and their implementation. The due dates for each tutorial is as follows:

  • McCulloch-Pitts neurons: 09/22
  • Single layer perceptrons: 10/06
  • Multi-layer perceptrons: 10/29
  • Unsupervised networks: 11/12
  • Deep neural networks: 12/02

Grading Scale

Grade Percentage Grade Percentage
A+ >95% B- 77-79%
A 90-94% C+ 74-76%
A- 87-89% C 70-73%
B+ 84-86% D 50-72%
B 80-83% F <50

How to Succeed in this Course

Reach out when you need support

Work towards dismantling previous ‘math trauma’

Unfortunately, math can be a topic that brings up angst and trauma. It certainly was for me as a student! I want us to work to dismantle the culture of “math people” and “non-math people” — we are all people who think mathematically.

This class is structured to be appropriate to those with a variety of formal backgrounds, and it is, therefore, more conceptual than mathematical. I want you to understand and be able to explain what the various techniques and models do, not necessarily conduct mathematical proofs that demonstrate how they work. The most important thing to remember is: ask lots of questions! I am more than willing to repeat or slow down if things get difficult or to explain it from another direction.

Collaboration

Collaboration is the basis of all scientific discovery and is instrumental in the learning process. I strongly encourage students to work on class notebooks and labs together if possible. If you are working with other students on a class notebook or lab, you must give written credit to the collaborators, and the short answer questions must be in your own words.

Class Policies:

Communication

Class communications will take place via Bates email. Please consider your Bates email to be the default place to look for class-related information and get into the habit of checking it daily. Your email messages to me will receive a response within 24 hours.

Commitment to Equity and Diversity

Everyone has the right to be treated with dignity and respect in our classroom, and I am committed to making that happen. But this not only means your instructor treating you with respect, but also you treating each other that way. Disrespect or discrimination on any basis will not be tolerated. Whether inside or outside the classroom, if you encounter sexual harassment, sexual violence, or discrimination based on race, color, religion, age, national origin, ancestry, sex, sexual orientation, gender identity/expression, or disability, you are encouraged to report it to Gwen Lexow, Director of Title IX and Civil Rights Compliance at Bates at or 207-786-6445. Additionally, please remember that Bates faculty are concerned about your well-being and development, and we are available to discuss any concerns you have. Students should be aware that faculty are legally obligated to share disclosures of sexual violence, sexual harassment, relationship violence, and stalking with the college’s Title IX Officer to help ensure that your safety and welfare are being addressed.

Academic Integrity

Cheating is bad, I think we can all agree to that. The less acknowledged truth is that it’s not even worth it. Cheating cheapens the value of your work and everyone else’s, and a single violation can literally ruin your entire academic and professional career. Please remind yourself of the Bates College policy on academic integrity and its definitions of plagiarism, use/misuse of sources, and cheating. Students’ work will be closely scrutinized for plagiarism and violations of the College policy will not be tolerated. If you are concerned that your collaboration might put you at risk of an academic integrity violation, please come see me during office hours as soon as possible. In my experience, violations of academic integrity are acts of desperation. If you are ever feeling desperate enough that a few extra points in this course seem to be worth risking so much, please consider talking to someone first — that could be me, a friend, or even someone at CAPS. I want you to succeed, and am happy to talk to you if you are feeling undue pressure from this course or anything else.

Students with Learning Differences

If you think you need accommodation for a disability or learning difference, please let me know as soon as possible. Some aspects of this course may be modified to facilitate your participation and progress. You will need to create documentation with the Office of the Dean of Students. I will treat any information about your disability with the utmost discretion.

Late work

The following policies apply to the following course components:

  • Reading annotations cannot be made up. Your two lowest grades will be dropped to account for any personal emergencies.
  • Colab notebooks cannot be made up. Your two lowest grades will be dropped to account for any personal emergencies.
  • Late tutorials will be subject to a 10% loss of points per day. In other words, a project that would have received a 100% grade will receive 90% after one late day, 80% after two late days, etc.

Emergencies

If I must cancel a class due to weather or an emergency, I will inform you via the class email list. Please consider your Bates email to be the default place to look for class-related information and get into the habit of checking it daily.

Because this class takes place in a pandemic…

The ongoing Covid-19 pandemic necessitates flexibility from everyone for this course to take place in a safe and supportive environment. Below are some key considerations:

  • To ensure the safety of other students and in line with Bates College and CDC health policies, everyone must wear a mask in the classroom at all times. Please ensure that your mask covers both your nose and mouth.

  • Some students may need to attend class remotely on a temporary basis due to 1) testing positive for Covid-19, 2) being in close contact with someone who tested positive for Covid-19, or 3) experiencing symptoms of Covid-19 or are otherwise not feeling well. I encourage you to err on the side of caution and attend remotely. If you need to attend class remotely, please watch the live class session after it is posted on Lyceum and email me to schedule an office hours appointment to address any questions you may have about class content.

  • As your instructor, I am working to ensure your safety by being fully vaccinated. However, if I test positive for Covid-19, have symptoms of Covid-19, or need to quarantine after spending time with someone who tests positive for Covid-19, the class will temporarily be moved to remote format via Zoom.

  • I recognize that this may all seem very unfair. It is. You did not ask for a global crisis, and you may personally have been doing everything possible to do your part. You may be experiencing stress in other ways, such as if someone close to you becomes ill. If you find yourself struggling, please come talk to me. I will help support you and point you towards additional resources.

Course Calendar

Unit 1: McCulloch-Pitts Neurons


September 2: Course introduction and introduction to artificial creativity

September 7: What’s neural about a neural network?

September 9: Boolean algebra: the basis of logical thought?

September 14: McCulloch Pitts neurons

September 16: How I stopped worrying and learned to love the dot product

Unit 2: Single-Layer Perceptrons

 

September 23: Perceptrons

September 28: Perceptron learning rule, part 1

September 30: Perceptron learning rule, part 2

October 5: Are single layer perceptrons any good?

Unit 3: Multi-layer neural networks


October 7: Build it deeper: the forward pass

October 12: Backpropagation 1: error surfaces and gradients

October 14: NO CLASS: FALL BREAK


October 19: Backpropagation part 2: hidden layers

October 21: Backpropagation part 3: four foundational equations

October 26: Backpropagation part 4: putting it all together

Unit 4: Unsupervised learning


October 28: Supervised versus unsupervised learning

November 2: Hebbian networks, part 1

November 4: Hebbian networks, part 2

November 9: Dataset bias and what to do about it

November 11: Convolutional neural networks for vision, part 1

Unit 5: Deep neural networks


November 16: Convolutional neural networks for vision, part 2

November 18: No class

 

November 23: NO CLASS: THANKSGIVING BREAK


November 25: NO CLASS: THANKSGIVING BREAK


November 30: Things we still don’t understand about deep networks

December 2: What is the relationship between deep networks and the brain?