The last four weeks of the semester will focus on synthesizing the concepts we’ve covered up to this point in the course. This will include some advanced topics for in-class discussion, but the course project will be the primary vehicle for you to demonstrate your understanding of the course material. You will propose, implement, evaluate, and present a system that builds on topics from the course. We will work as a class to identify key topics from the course and to discuss potential projects.
Your final grade on the project will be based on four components:
You can read about each of the projects components below.
You will begin your work on the course project with a proposal. The purpose of this proposal is to demonstrate that you have thought about how your project will connect important ideas from the class, how you will complete your work on the project, and how you will evaluate your work. The proposal will take the form of a short written document, which you will have a few days to prepare.
We’ve explored a wide variety of ideas and technical skills in this course, so there are many possible directions you could take your project. Regardless of the project you choose, you must clearly state a goal, and then combine at least three different concepts we’ve covered in this class in a meaningful way. Concepts from class include, but are not limited to:
Your project does not need to make equal use of all three concepts, but should combine them in an interesting and unexpected way.
As an example, writing a program that runs in a process, opens a file, and uses mmap to put it in your address space would not be a particularly interesting project and would likely not be approved.
However, a project that uses virtual memory techniques to catch use-after-free errors and records them in a human-readable file format would be an interesting application of these concepts.
I may be willing to approve projects that draw from fewer than three concepts from class, provided they are sufficiently creative and technically demanding. You will need to discuss this possible project with me before submitting your proposal.
One concern to keep in mind as you think about your project is that you will need to evaluate your system in your final report on the project. As you work on your projects we will look at several research projects, all of which contain an experimental evaluation. This may influence your plans for an evaluation, but at this stage you should make sure you select a project where there is an interesting quantity to measure, such as runtime performance, memory use, network time, bug detection rates, etc. Ideally, this quantity should be closely connected with how well your system is working.
You may work on your project individually, or in groups of up to three students. We will spend some time in class working to identify areas of common interest so you can make informed decisions about the group you work with.
Your project proposal must:
While you are not required to use figures in your proposal, they may make it easier to explain your plans. Your approach section should describe the major components of the system and how they will interact, while the implementation section goes into detail about how these components will work. You may want to consider possible alternate solutions if your proposed strategy does not work. Likewise, your evaluation plans should discuss what you can measure or assess if your project progresses as planned. Your proposal should be 2–3 pages of single-spaced text with a reasonable font.
The largest component of this project is the actual implementation. You are free to divide up the implementation work as you see fit, and you are welcome to use the language(s) of your choice, provided you stay within the vicinity of your proposed project. Every group member must play a significant role in the implementation, but you are welcome to divide up implementation tasks.
Your implementation will be evaluated using several criteria:
While the first three pieces should be easy to evaluate, it may be difficult to say whether your system works. Two weeks is a very short time for a large implementation project, so I understand that you may run into issues and your plans could change. I have tried to offer advice about which parts you could drop if you run into issues in my proposal feedback, but if you hit a major roadblock and need to change your project I am happy to help.
To show that your system works, you must provide a README file that precisely describes the basic use of your system, including both general instructions and a specific example that walks through an interaction with the system and the expected output.
Your implementation must also include all necessary source and data files and a Makefile that allows me to build your project with the make command.
Your report must give an overview of your system, describe the design and implementation of the system, and finally present an evaluation of your work. This report should cover all of the information from your in-class presentation, with an additional level of detail. A report that earns an A would include sufficient detail for me to re-implement your system without seeing your code or presentation, while also conveying the high-level ideas underlying your system.
Your report will need to include the following sections:
Give a brief overview of your system’s design, the high level points of your implementation, and summarize your evaluation strategy and results. Your overview should not exceed 500 words.
The design and implementation section should explain the structure of your system and, importantly, the rationale for that structure. Design decisions include details like the data structures and algorithms you used, the places where you used concurrency or decided to run code on the GPU, and any libraries you may have used for your implementation.
Try to organize this section in a top down manner: introduce the major components of your system, outline the responsibilities of each component, and then jump into the specifics of each component. You may want to repeat this process for sub-components if there are reasonable logical divisons within the major pieces of your project.
Another strategy for organizing this section would be to break it down by concerns. System concerns include updating a file index in response to a file deletion, maintaining game state over a network connection, or other general requirements that may not map to specific units of code in your system. If you can identify a handful of high-level concerns it may make more sense to discuss the overall implementation structure and then describe how each concern fits into this structure.
Your design section should be approximately two pages of text, plus figures where appropriate.
You are required to present an evaluation of your system.
The details of this evaluation will depend on your project topic, but there are a few common requirements.
First, you must describe the experimental set-up;
this includes the hardware and software you are using for the evaluation, versions of any important software tools (including libraries), and the methods you use for gathering data (e.g. we measure execution time using the time command).
Then, you should measure an appropriate aspect of your system’s behavior while varying some aspect of the load on your system or its environment.
Make sure your evaluation section explains what you are trying to measure and discusses an interpretation of your results.
If you are unsure what you can measure about your intended project I am happy to discuss possible approaches with you.
When appropriate, your evaluation should include graphs to show the results of your evaluation. Graphs may not be appropriate for all evaluations, so feel free to be creative in how you present the findings from your evaluation.
Presentation times will depend on the number of project groups, so check back for more details. Regardless of the available time, presentations must answer the following questions:
While you do not need to go into detail about the code you wrote for your implementation, you should try to give us enough information that we understand both the conceptual design of your system and some of the interesting technical challenges you faced during your implementation.
For part three, it’s not enough just to report that your system was successful. You should show us your evaluation, discuss the results, and argue for your interpretation of these results. Remember to relate the evaluation back to the original purpose of your system.
All group members must participate in the presentation, but it is up to you to decide how to divide the presentation topics.