Assessment
1 Assessment overview1
This course will be assessed on the basis of a two summative assessments:
A final self-reflection of your learning during the course (500 words, 20% of the final mark).
A group project (80% of the final mark, no word limit). The group project must be a data analysis report.
During the course, you will also have the chance to practice your skills and test your knowledge through formative assessments. While the formative assessments do not contribute to the course mark, you are expected to complete them.
See the sections below for more information on feedback and assessment.
For any question about assessment, post your question on Piazza (unless it’s of a sensitive nature, then get in touch with Stefano).
2 Formative assessments
You have the chance to practice your skills and test your knowledge through formative assessments. While the formative assessments do not contribute to the course mark, you are expected to complete them. There will be five formative assessments:
Self-reflection of weeks 2-3. Due Week 4.
Quiz of weeks 1-5. Due Week 6.
Self-reflection of weeks 4-5-6. Due Week 7.
Self-reflection of weeks 7-8-9. Due Week 10.
Quiz of weeks 1-10. Due Week 11.
2.1 Self-reflections
Learning is an active process: it happens best when we’re all engaged. Learning is also something that we can learn how to do better. Think of a hobby you might have or a skill you’ve learned: you’ve probably perfected it for a while, paying close attention to what did or didn’t work.
That’s why both students and the lecturer (Stefano) will reflect on how the course is going and what they’ve learned. At the defined check-point weeks (weeks 4, 6, 10), we’ll all write short self-reflections. You will receive feedback on yours and you’ll be able to comment on mine. You will submit your self-reflections to Turnitin on Learn, while I will post mine on my Reflective Journal.
In your self-reflection, you want to focus on your learning process: what did you learn, how easy or difficult was it, how did the learning affect your views on something? Here are more specific questions you can follow, but feel free to make adjustments. We recommend to lay out your the self-reflection based on the questions (see examples below) and to add a title to the self-reflection in the format “Weeks 2-3”, “Weeks 4-5-6”, and “Weeks 7-8-9”. See the Overview box above for how much you should be writing in each self-reflection.
The following boxes show three strong examples and three weak examples of self-reflections.
When writing your reflections, make sure they are reflective! I know it sounds silly, but we have seen several students just writing a dry list of what they have done (and in most cases this is simply a list of what you are expected to do).
2.2 Feedback on self-reflections
After you submit your self-reflections to Turnitin on Learn, you will be given feedback on what is good and what can be improved in your self-reflection. We aim to release feedback within one week after the submission is due.
Note that we are not expecting beautiful prose, rather we will be looking for honest reflection. Feel free to add specific questions about aspects of your reflection in the reflection itself for me or the tutors to reply in our feedback.
2.3 Quizzes
The two quizzes will be online tests with multiple choice and true/false questions. They will be available on Learn.
Quiz of weeks 1-5. Due Thursday at noon Week 6.
Quiz of weeks 1-10. Due Thursday at noon Week 11.
3 Summative assessments: Final Reflection (20%) and Group Project (80%)
3.1 Final self-reflection (500 words maximum, 20%)
For the Final Self-reflection, you should write a maximum of 500 words. In the Final Reflection, you should reflect on your learning experience throughout the course. You can think of the Final Reflection as the synthesis of the individual formative Self-reflections (but do not just copy paste what you have written in them; you should write a synthesis of your overall experience rather than just a list). Below you can find questions to guide you, like for the formative Self-reflections. We recommend that you structure your reflection using the headings of the questions below as headings in your Final Reflection.
3.2 Group Project: data analysis report (no word limit, 80%)
As part of the assessment you will have to work on a Group Project. The group project must be a data analysis report, but you can choose the topic as long as it is in linguistics (see below for details). Your project has to be approved by Stefano (see Section 3.3 below) and you must use the project template (see requisites below). Note that there is no word limit for the Group Project.
A data analysis report is a special type of document which resembles an academic paper, but focuses more on the analysis aspects of the study. This means that your report should include visible R code, more plots than what you would find in a traditional paper, clear explanation of your analytical choices, and so on. While you are required to write a literature background that justifies your study, this should be brief and targeted (as short as possible but as detailed as necessary). References to relevant literature should be included, but there is no need to discuss everything in detail. Only include information that is required for understanding your research question/hypotheses.
There are four main types of project that are appropriate for the data analysis report format.
The Group Project has the following further requisites.
Background. This should be brief (2-3 paragraphs) but should discuss relevant literature in enough detail to justify the research questions/hypotheses.
Research Questions/Hypotheses. The research questions should be defined precisely and it should be clear how they follow from previous work discussed in the Background. If you wish, you can also define precise research hypotheses (it is fine to not have hypotheses). Your RQ/RHs don’t have to be novel/original. A simple, well-justified RQ will be better than many complex vaguely defined RHs.
Methods. The sections under Methods should be as detailed as possible to allow an independent research to exactly repeat the study. Materials and procedures, if applicable, should be described. If the project uses pre-existing data or simulates data, this should be clearly stated and details should be provided.
Analysis plan. The planned statistical analysis must be justified and care should be given ensuring that it straightforwardly addresses the RQ/RHs. A description of which statistical coefficients/comparisons will be used for which RQ/RH should be included in the report.
Data processing. Appropriate processing should be applied to the data, including data cleaning and data validation checks. Processing steps should be justified where relevant (for example, when filtering data or recoding labels).
Summaries. Appropriate summary measures (central tendency and dispersion) must be provided for relevant data and data groupings, as formatted tables. A brief description should accompany the measures.
Plotting. Appropriate plots illustrate patterns and characteristics of the data. Plots should be accompanied by a caption and a textual description. Visualisation principles of clarity and honesty should be followed.
Modelling. You must carefully justify the specification of regression models with appropriately selected variables, family distributions, interactions if applicable. When multiple models are necessary, you must explain why and how each model is linked with which RQ/RH.
R/Quarto skills. These are general R and Quarto/markdown skills not covered in the other areas. They include general code/text organisation, markdown formatting, use of headings, code chunk labels, referencing, code commenting, code succinctness and clarity.
Open Research. The Research Compendium of the study must be available online. We will assess file/folder structure and organisation, clarity of documentation, including degree of data documentation, and whether code runs.
3.3 Project proposal submission
- You will have to submit a very short project proposal for approval by Thursday 5th November to Turnitin on Learn, but the earlier you submit the earlier you can start working on it! We will be checking submissions as they arrive and ask to get in touch if there are any issues.
- Only one person per group should submit the proposal (there is no need for everyone in the group to submit the project proposal and you should just add all the exam numbers of the group members).
- A few sentences explaining the project will be sufficient. Note that this deadline is an informal deadline and there is no possibility of getting an extension.
- The submission will be marked as such: a “1” means you can go ahead with the project. A “0” means you should book office hours with Stefano to discuss alternatives or adjustments. You might also find some textual feedback attached to the submission.
The following boxes give you some examples of possible projects and project proposals.
3.4 Group Project and Final Reflection submission
- You can use any file format for the Final Reflection, but you must submit a PDF file for the Group Project generated from the template.
- XXX
4 Feedback and marking
Feedback will be provided to you (1) during class, (2) on your weekly self-reflections and (3) during office hours (it is up to you to book meetings with me; you can do so here: https://bit.ly/33BH84L), (4) on the Final Reflection + Group Project submission.
Marking will follow the approved PPLS/LEL marking scheme, which you can find on the PPLS Undergraduate Hub and the PPLS MSc Hub on SharePoint. For feedback, we will use the feedback forms reported in the following sections. These forms have been created to streamline feedback and to allow you to understand which areas of the assessment were strong and which could have been improved. The feedback forms will help us coming up with a mark, but note that there is no mathematical formula which translates the feedback form to a numeric mark.
For the numeric mark, we will limit marks to the following values: 0, 15, 25, 35, 45, 55, 65, 75, 85, 95. For students who are not used to the marking system of the PPLS School, we would like to stress that 65 is already a very good mark, 75 is excellent and 85 or above are outstanding (above what expected from even the best student at their level).
4.1 Feedback rubric for Self-Reflections and Final Reflection
The following feedback rubric will be used for feedback on the formative Self-Reflections and the summative Final Reflection.
| Excellent | Competent | Limited | Absent | |
|---|---|---|---|---|
| Engagement with quantitative methods | Excellent critical engagement with research methods and statistical concepts. A range of methods and concepts from the course have been covered and meaningfully connected. | Good or very good engagement with research methods and statistical concepts, although it is descriptive to partially critical. A range of methods and concepts are discussed, although there might be limited critical connection. | Limited engagement with research methods and statistical concepts. When present it is only descriptive. Only a very limited range of methods and concepts are discussed. | Very limited or totally absent engagement with research methods and statistical concepts. |
| Depth of reflection and learning | Critical reflection on engagement in the course and the learning process. Excellent discussion of impact of learning on own views and understanding of quantitative methods and beyond. | Good or very good level of reflection on engagement in the course and the learning process. Impact of learning on views and understanding of quantitative methods is mostly descriptive, with minimal insight and limited connection to concepts beyond those treated in the course. | Limited level of reflection on engagement in the course and the learning process. Impact of learning on views and understanding of quantitative methods is only descriptive, with no insight and limited connection to concepts beyong those treated in the course. | Very limited or totally absent reflection. |
| Use of evidence | An excellent variety of specific examples are provided and critically discussed to illustrate engagement and impact on understanding of quantitative methods. | Good or very good selection of examples to illustrate engagement and impact on understanding of quantitative methods. | Limited selection of examples to illustrate engagement and impact on understanding of quantitative methods. | Very few or no meaningful examples. |
4.2 Feedback rubric for Group Project
The following areas in the data analysis report (Group project) will be assessed.
Research framing: positioning of the study within its academic context, developing a clear rationale, and formulating well-justified research questions or hypotheses that align with appropriate methods and analyses.
Summaries: describing and presenting data through accurate measures of central tendency and variability, using clear tables and interpretations to identify key patterns, distributions, and differences across variables.
Plotting: effectively presenting data through appropriate plots and clear visual design, using accurate and transparent representations to highlight key patterns, relationships, and features that support meaningful interpretation.
Modelling: selecting, justifying, and implementing appropriate statistical models that address research questions or hypotheses, with careful consideration of variables, assumptions, distributions, interactions, and the coherence of the overall analytical strategy.
R/Quarto: using R and Quarto/Markdown effectively to create well-structured, clear, and transparent documents, with organised code, appropriate formatting, and concise, well-justified analytical workflows.
Open Research: organising, documenting, and sharing research materials in an open, reproducible, and reusable format, with clear file structures, comprehensive documentation, and workflows that support transparency and reproducibility.
The following feedback rubric will be used for feedback on the Group Project.
| Excellent | Competent | Developing | Absent | |
|---|---|---|---|---|
| Research framing | The discussion of relevant literature demonstrates an excellent and sophisticated understanding of the topic area, with sustained critical engagement and effective synthesis of previous work. The rationale for the study is convincingly developed and demonstrates clear insight into theoretical and methodological issues. Research questions and/or hypotheses are exceptionally clear, focused, and strongly justified by the literature review. The report demonstrates an excellent understanding of methodological principles. | The discussion of the relevant literature demonstrates a good or very good understanding of the topic area, with some critical engagement. The rationale for the study is clearly developed and linked to previous work. Research questions and/or hypotheses are clear and justified by the literature review. They are appropriately formulated and connected to the planned analyses. The report demonstrates a good or very good understanding of methodological principles. | The discussion of the relevant literature is largely descriptive, demonstrating a minimal understanding of the topic area. The rationale for the study is only partially developed and shows weak links to previous work. Research questions and/or hypotheses are present but may be unclear, only loosely connected to the literature review, or insufficiently justified. Important aspects of the literature are overlooked or discussed superficially, and the overall formulation requires substantial improvement. The report demonstrates a weak understanding of methodological principles. | The discussion of the relevant literature is absent or so limited that it demonstrates negligible understanding of the topic area. There is no meaningful rationale for the study, or any rationale provided is unsupported and unrelated to previous work. Research questions and/or hypotheses are absent, unclear, or lack any meaningful justification from the literature. The literature review is missing, extremely superficial, or largely irrelevant. The report demonstrates little or no understanding of methodological principles. |
| Summaries | Summary measures are highly appropriate, comprehensive, and expertly implemented, with correct use of measures of central tendency and dispersion for all relevant variables. Summary tables are clear and well designed. Descriptions demonstrate insightful interpretation of patterns, linking descriptive findings meaningfully to the research questions. | Summary measures are generally appropriate and competently implemented, with correct use of central tendency and dispersion for all relevant variables. Summary tables are clear and well formatted. Descriptions identify the main patterns and variability with depth of interpretation or comparison. | Summary measures are included but are not always appropriate or correctly implemented. Measures of central tendency and dispersion may be missing, incorrectly applied, or incomplete for some relevant variables. Summary tables are present but have notable weaknesses in formatting, clarity, or completeness. Descriptions identify only the most obvious patterns with superficial interpretation and little consideration of variability or comparison. | Summary measures are absent or almost entirely inappropriate or incorrect. Measures of central tendency and dispersion are missing or incorrectly applied throughout. Summary tables are absent or so incomplete, inaccurate, or poorly presented that they do not meaningfully summarise the data. Descriptions of the data are absent or fail to identify even the most basic patterns. |
| Plotting | Plots are highly effective in communicating key data patterns and relationships. Plot types are carefully selected and demonstrate excellent judgement in relation to the data and research questions. Visual design is clear, accurate, and sophisticated, enhancing interpretation while avoiding unnecessary complexity. Captions are informative, precise, and allow figures to be understood independently. Accompanying descriptions provide insightful interpretation of important patterns, demonstrating a strong understanding of visualisations principles. | Plots are well designed and effective in illustrating the main data patterns and relationships. Appropriate plot types are selected, and the visual design is clear, accurate, and supports interpretation of the data. Captions are informative and provide sufficient context for understanding the figures. Accompanying descriptions accurately identify and explain the main patterns and relationships, demonstrating a good understanding of the information conveyed by the visualisations. | Plots are included but are not always appropriate or effective in illustrating the main data patterns. Plot types may be poorly chosen or inconsistently applied, and aspects of the visual design reduce clarity. Captions are brief or incomplete and are generally not understandable as standalone descriptions. Accompanying text identifies only the most obvious features of the data with little interpretation. | Plots are absent or almost entirely inappropriate for the data. Figures, if included, are poorly constructed, misleading, or uninterpretable. Captions are absent or provide no meaningful information. Accompanying descriptions are absent or fail to describe the data in a meaningful way. |
| Modelling | Regression models are exceptionally well justified and demonstrate sophisticated understanding of the research questions and analytical aims. Variable selection is carefully considered and critically evaluated. Distributional assumptions are thoroughly assessed, and modelling choices are clearly justified. Interactions, where included, are theoretically or empirically well motivated. Where multiple models are used, they form a coherent and well-integrated analytical strategy. | Regression models are appropriate and clearly linked to the research questions. Variable selection is well considered and generally justified. Distributional assumptions are appropriately assessed and suitable modelling choices are made. Interactions, where included, are appropriate and supported by reasonable theoretical or empirical justification. Where multiple models are used, they are clearly explained and contribute to a coherent analytical strategy. | Regression models are included but show only a limited connection to the research questions, with little or no justification for modelling choices. Variable selection is only partly appropriate and is weakly motivated. Distributional assumptions may be inadequately assessed or incorrectly justified. Interactions, if included, lack clear theoretical or empirical motivation. Multiple models, where used, show weak coherence or unclear purpose. | Regression models are absent or entirely inappropriate for the research questions. There is no meaningful justification for modelling choices, variable selection, or model assumptions. Distributional assumptions are ignored or fundamentally misunderstood. Interactions, if included, are inappropriate or incorrect. Multiple models, where present, show no coherent purpose or interpretation. |
| R/Quarto | R and Quarto/Markdown practice demonstrates excellent computational, reporting, and reproducibility skills. The document is exceptionally well structured, with a clear and intuitive organisation. Code chunks are appropriately labelled, efficiently organised, and integrated effectively. Markdown formatting is used skilfully to enhance readability and presentation. Code is clear, concise, well documented, and demonstrates strong attention to reproducible analytical practice. | R and Quarto/Markdown practice is effective and demonstrates good computational and reporting skills. The document is well structured, with clear headings, appropriate formatting, and a logical organisation. Code chunks are clearly labelled and organised. Markdown formatting is used appropriately to support readability. Code is clear, well structured, and supports reproducible analysis. | R and Quarto/Markdown practice is basic and possibly inconsistent. The document has some structure but may be difficult to follow due to weak organisation or formatting. Headings, code chunk labels, and Markdown features are used inconsistently or incorrectly. Code is functional in places but lacks clarity, organisation, or readability. | R and Quarto/Markdown practice is absent or demonstrates negligible competence. The document lacks meaningful structure or formatting. Code chunks are absent, disorganised, or unusable. Markdown is not used appropriately. Code is absent, largely non-functional, or so poorly organised that it cannot be meaningfully evaluated. |
| Open Research | The research compendium is exceptionally well organised, fully documented, and designed to maximise transparency. File and folder organisation follows best practice and enables intuitive navigation of all project materials. Documentation is comprehensive and clearly describes data provenance, processing steps, analytical decisions, and workflow procedures. Code runs successfully in a clean environment and reproduces reported results without manual intervention. The workflow demonstrates an outstanding commitment to open research principles. | The research compendium is well organised and clearly structured, supporting navigation and reuse. File organisation is appropriate and facilitates access to project materials. Documentation is provided and includes relevant information about data provenance, processing steps, and workflow decisions. Code is structured to support reproducibility and runs successfully in a clean environment with minimal intervention. The workflow demonstrates a good commitment to reproducibility and open research principles. | The research compendium demonstrates only a minimal level of organisation. File structure is incomplete or difficult to navigate, reducing usability and opportunities for reuse. Documentation is minimal or incomplete, with important details regarding provenance or processing steps omitted. Code may not run successfully in a clean environment without substantial manual intervention. The workflow demonstrates weak attention to reproducibility and open research principles. | The research compendium is absent or unusable. File organisation is missing or so poor that the project cannot be navigated or reused. Documentation is absent or provides no meaningful information. Code does not run or cannot be evaluated in a clean environment. There is no meaningful evidence of reproducibility or adherence to open research principles. |
References
Footnotes
Some of the text on this page is from Itamar Kastner’s Morphology course site.↩︎