Project

As part of this class, you will conduct a small-scale systems research project. While your project will necessarily be smaller than the multi-year efforts that lead to some of the papers we read, it will have many of the same elements as a larger scale project. You can work individually on projects, or in groups of up to three students.

Project Requirements

Your project will need to satisfy the following requirements:

Identify a Problem: You will need to select an interesting problem from systems research that you can focus on. We will only have five weeks to work on projects, so please think about how you can scale down the problem to a size that is manageable.
Propose a Solution: Propose a solution to the problem you have identified. You are welcome to choose solutions that mirror work from published research, or you can design your own approach to a problem. Your solution must include at least one of the techniques (i.e. ptrace, library interposition, optimistic concurrency control, etc.) we have read about or discussed in the systems research literature. If you want to choose a technique from systems research that we have not read about, please discuss it with me first.
Implement your Solution: Turn your approach into a working system. This is where you will spend much of your time. It is a good idea to think through how you could implement an approach before you commit to it, and even better to have several possible routes to a solution in mind.
Evaluate your Solution: Test your system. That means making sure it works in the cases you expect it to, but you will also need to measure the system in some meaningful way. Asses its overhead, the specific issues it can solve, the cases where it cannot solve a problem, and so on. You will need to spend some time thinking about how you can evaluate your system; you might want to think about this process early so you can focus your implementation effort on building a system with all the pieces you need to conduct an evaluation.
Present your Work: Give a short presentation about the problem, the solution you implemented, and the results of your evaluation. Presentations will take place during our scheduled final exam time.

Selecting a Project

You do not need to write a formal project proposal. Instead, I will meet with each group during our project working days and ask you to describe your project and how it meets the requirements. I do recommend keeping notes as you plan your project work so you can easily refer back to your goals for the work. I may encourage you to focus on specific strategies or adjust your project scope if I am concerned your work may not meet the requirements or would take too long to complete.

Problems in Systems

What are some basic, open problems in systems research?

Concurrent programs have special bugs. How can we check for them?
Memory errors are bad. How can we find and/or prevent them?
Systems crash. How do we tolerate that?
Distributed systems create parallel timelines. How can distributed components agree on anything?
Programs that run slowly are bad. What information should a programmer be able to get so they can improve their programs?
Programs are insecure. How can we make them secure?
Increasing the scale of a system changes its characteristics. What trade-offs make sense at larger (or smaller) scales?
It’s hard to write programs with threads. What is the right paradigm to implement a parallel program?
It’s hard to understand parallel programs’ performance. How can we find the bottlenecks so we can fix them?
Debugging is hard. How can we improve debugging tools?
Error messages are confusing, especially when they’re about types. Can we give better information when a program does not compile?
It is hard to visualize large linked data structures. Can a tool help with this?
Reverse engineering is hard. Can we learn something about a program from its machine code, memory, etc?
Managing networks to distribute load and avoid bottlenecks is hard. How can we help?
Storing data reliably is important. Can we build fast, reliable storage systems?
Storing multiple versions of a file when multiple users are editing them is hard. Can we do better than git?
Programmers struggle to understand where memory lives in a program’s address space. Can we give better information about this?
It’s hard to manage memory correctly in C. Can we provide useful tools to make it easier, or fix/catch problems?
Lots of systems use key-value stores. How can we make them better (e.g. fast, reliable, etc.)?
Network access is controlled within some countries. Can systems help people in these countries access the outside world, possibly anonymously?
Little annoyances get in the way of the software developer workflow. Can we improve the tools developers use?

Most of these are large, general issues. I recommend that you focus on a narrower context where concerns like those listed above would make sense.

Implementation

The largest component of this project is the actual implementation. You are required to complete your implementation primarily in a systems language; C, C++, CUDA, Rust, and Go all fit under this definition, but I am open to other options if you discuss them with me. If building some pieces of your project in another language will save you time, that is perfectly fine to include as a small part of your larger implementation in a systems language.

Every group member must play a significant role in the implementation, but you are welcome to divide up implementation tasks. If you have difficulty working or coordinating with your group, please talk to me soon so I can help.

Regardless of the language you choose, make sure you write clear, well-commented code. Your code must build without warnings or errors, and should include runtime checks for errors or failures when anything can go wrong. Include a README file that precisely describes the basic use of your system, including both general instructions and a specific example that walks through an interaction with the system and the expected output. Your implementation must also include all necessary source and data files and a Makefile that allows me to build your project with the make command. If building or running your project requires any additional software or configuration beyond what is available on MathLAN, please make sure to document these dependencies so I will be able to run your project.

Evaluation

Once you have an implementation of your system, you will need to evaluate it. You will need to plan and conduct a experiment that allows you to show us that your system works, but the nature of this experiment will depend on the problem and your approach to solving it. Make sure the experiments you run align with the problem you set out to solve.

I recommend that you think ahead to how you will evaluate your system so you can plan your implementation around building just the necessary features to run your experiments. I am happy to offer feedback and advice about evaluation strategies, and you should expect questions about your plans for evaluation fairly early in the process.

Presentation

Finally, every group will present their project. Presentations must answer the following questions:

What is the problem you are trying to solve?
Why is this problem important?
What is the high-level approach you are taking with your solution?
How did you implement that approach?
How did you evaluate your system and what did you find?

While you do not need to go into detail about the code you wrote for your implementation, you should try to give us enough information that we understand both the conceptual design of your system and some of the interesting technical challenges you faced during your implementation.

For part five, it’s not enough just to report that your system works. You should describe your evaluation procedures, show results, and walk through how we should interpret them. Remember to relate the evaluation back to the original purpose of your system.

All group members must participate in the presentation, but it is up to you to decide how to divide the presentation topics.