Re-EXAM August 2026 - Due date August 23rd 23:59

Please read this section carefully! It states how to approach the exam, how to finalize the exam and expectations!

The take-home exam comprises five questions.

A pass on answers to Questions 1-4 results in grade 3/G. If you don't pass these questions, Question 5 does not contribute to your grade so make sure you do well on the first 4 questions before attempting 5!

For grade for 4 you need to pass with distinction on two of Questions 1-4, pass on the remaining Questions 1-4, and pass on Question 5.

For a VG you need pass with distinction on three of Questions 1-4, pass on the remaining Questions 1-4, and pass on Question 5.

For grade 5 you need to have pass with distinction on Questions 1-5.

What are the expectations?

• Pass: The submitted work demonstrates that the student can independently perform a scientifically sound analysis of the assigned dataset. Methods are appropriate and motivations for choices made in constructing pipelines etc are provided, conclusions are supported by evidence, simulation results are used meaningfully, and limitations and sources of uncertainty are acknowledged.
• Pass with Distinction: The submitted work demonstrates a deep understanding of the dataset and methods used. Analyses are well justified, results are interpreted critically, simulation results are integrated into the conclusions, limitations are discussed in depth, and the work shows insight beyond the direct application of standard methods.
• Not Pass: The submitted work does not sufficiently demonstrate the intended competencies. Analyses are incomplete or inappropriate, conclusions are unsupported by evidence, important methodological issues are overlooked, or simulation studies are missing or inadequate. Choices made in constructing pipelines etc lack clear motivation.

Note! Grades are based on the quality of the methodology, reasoning, interpretation, and supporting evidence, rather than on obtaining specific numerical results or identifying the exact data-generating process.

I will give partial credit so it's always worth while attempting each task. However, credit is awarded for analysis, interpretation, and motivation of conclusions, not for presenting figures, tables, or output without discussion. Methodological errors that cause information leakage (e.g., feature selection, imputation, scaling, or tuning performed using validation or test data) will result in substantial loss of credit.

You should work alone for this exam. The reports will be checked with Urkund.

"Can I use AI for coding?" It's been my experience that, while AI can be helpful, it is also overconfident in providing answers that are in fact wrong. It takes time to check the output from AI and, depending on your own skillset, this may result in an overall waste of time rather than saving time, especially if you want to connect multiple tasks. Your main task is to analyze and interpret the results. AI might help you with setting up the analysis but less so in interpreting and spinning the analysis in the way I have requested. The key is insights and discussion as to why results look like they do.

So I would say that you should use AI with caution and only for simple coding questions like generating plots, proposed functions calls (which may contain bugs), etc. Any text submitted must reflect your own understanding and interpretation of the results. AI-generated explanations or discussions that are not critically checked and rewritten by you are unlikely to receive credit.

Note, you are working with different data sets so there is no single correct answer. Different datasets contain different challenges, and grading is based on the quality of the methodology, reasoning, interpretation, and supporting evidence.

GOOD LUCK!

• The report should not be longer than 1 page of text + max 1/2 page conclusion/main summary per question.
• You do not have to discuss the methods and assumptions unless this pertains to a result you present - i.e. you don't need to write a "lab report" with introduction, methods, analysis and conclusions. Here, it's only analysis and conclusions that I am interested in.
• Figures and Tables: maximum of 4 figures and 2 tables per question.
• Make sure you provide captions for the tables and figures - these should clearly state what the figure/table contains as well as a 1-2 conclusion statement about what can be learnt from the figure/table. Refer to the figures and tables in the text!
• MOTIVATE YOUR ANSWERS!
• Don't contradict your results! If something is unexpected say so and explain why you thought you would see something different.
• Please think about your presentation. Spell check, grammar check. "The arm-length check": meaning, skim through the report as it appears on the screen - does it look nice and neat or is it messy with mis-alignments, cut-off sentences, plots that are tiny etc?
• No cut and paste code mixed with the text.
• No, jupyter notebooks are not accepted as a report. Preferably use latex (overleaf).
• Yes, you can use bulletpoints to present your findings. This could be especially useful for the subtask conclusion.
• Stress your main findings in the 1/2 page per-question conclusion.
• !!!!Make sure you put the figures with each question after the subtask discussion and before the next question so I don't have to scroll back and forth between tasks when I grade.!!!

IMPORTANT

Wherever the exam task contains a question ("Does X differ from Y?") it is not sufficient to answer "Yes" or "No" - you need to explain how you arrived to this conclusion, which results provide you with these answers, as well as a statement trying explain why the results are what they are. Likewise, when a task is phrased as "Can you do X to get Y?" I, of course, mean for you to attempt to answer this question by doing something, not just hypothesizing about it.

To answer the questions it is for the most part inadequate to perform the task once on the full data set. Make use of re-sampling techniques to support your findings.

Data sets

You will be working on your own data sets that I have put on canvas. Everyone's data set is a bit different, by construction. The zip file contains three files: X_TR, y_TR and X_TE, that is, a training data with labels and a separate test set without labels. You should use the data set corresponding to the last digit of your social security number 1-9, 0=10. Example: if your social is xxxxxx-xxx2 you should use the data set located in file ND_2.zip.

All of the data sets contain 10 classes. Missing values are denoted by nan. Different datasets contain different combinations and severities of challenges. One objective of the exam is to identify and investigate the challenges present in your assigned dataset.

Now, let's get started!

NOTE!!!! For all Questions you should analyze the sample size (training) effect on the results.

NOTE!!!!!!!!!! Code reuse is encouraged. Many parts of the exam can be addressed using the same analysis pipeline. The questions differ not so much in the code required, but in the conclusions, interpretation, and insights expected from the analysis.

NOTE!!!!!!!!!!!!!!!!!!!!! All claims and conclusions should be supported by quantitative evidence. Qualitative arguments alone are insufficient.

Question 1

Construct a pipeline to analyze the X_TR, y_TR dataset. Motivate all major choices made in your pipeline. Investigate the effect of sample size on your results.

Make sure to discuss:

Overall and class-specific performance of methods and how you assessed this.

Why some methods perform better than others on this dataset and for different sample sizes, and how you assessed this.

Particular challenges in your dataset that your analysis identified, the evidence supporting their existence, and how you addressed them.

There is no single correct analysis pipeline. Your choice of methods should be motivated by the characteristics of your assigned dataset.

Question 2

Which features are related to the class labels? How many features are needed for effective classification? Explain how you assessed this, whether your conclusions differ between methods, and how sample size affects your conclusions.

Make sure to discuss:

the stability of feature selection and how you assessed it

particular challenges in your dataset that affect the feature-selection task and how you addressed them

the impact of feature selection on classification performance

whether different feature-selection methods lead to similar or different conclusions

Question 3

Identify the most important challenge affecting your dataset and analysis pipeline.

Using quantitative evidence, justify why this is the dominant challenge in your dataset. Modify the dataset to increase the severity of the challenge and re-evaluate your analysis pipeline on the modified dataset.

Make sure to quantify and discuss:

*the effect on classification performance

*the effect on feature-selection results

*how these effects change with sample size

*why the observed effects are expected given your modification

Clarification: The challenge should be specific to your dataset and not a generic deterioration such as adding random noise. Your task is to identify what makes your dataset difficult and increase the severity of that challenge. Explain why the chosen modification is appropriate and which methods you expect to be most affected.

Question 4

Apply your top-2 best performing methods to the test data X_TE. Can you say something about how confident you are in your predictions? Be aware that there may be a data drift in TE, a change in content distribution compared with TR.

Make sure to discuss:

Prediction confidence: How do you assess confidence in your predictions on X_TE? Support your approach by demonstrating how it performs on validation/test splits derived from X_TR, y_TR.

Prediction summary: Summarize the prediction results for both methods overall and at the class level. What do the results suggest?

Sample size: How are prediction confidence and expected performance affected by the training sample size?

Data drift: Is there evidence of data drift in TE compared to TR? Describe how you assessed this and discuss the implications for your predictions

Mislabeling: Is there evidence of mislabeling in the training data? How might such mislabeling affect the predictions and confidence estimates on X_TE?

Question 5 (for higher than passing grade only)

For your dataset, select one of the best-performing and one of the worst-performing methods. Construct a modification (disruption) of the dataset that causes the performance of the best-performing method to deteriorate more rapidly than that of the worst-performing method.

Re-evaluate both methods on the modified dataset.

Make sure to discuss:

how the dataset was modified and why the modification should affect the methods differently

whether the modification had the intended effect and how you assessed this

which characteristics of the methods explain the observed differences

how the relative performance changes with sample size

Clarification: The goal is not simply to make all methods perform worse. The goal is to construct a modification that specifically targets the strengths of the better-performing method, allowing the weaker-performing method to become relatively more competitive.