BUILDING ANALYTIC SKILLS IN UNDERGRADUATE BUSINESS STUDENTS: THE IMPACT OF PROJECT MATHS Anthony Cronin and Paula Carroll Quinn and Smurfit Schools of Business, University College Dublin, Ireland Abstract: This article seeks to explore the links between an Undergraduate Data Analysis for Business module and Project Maths (PM), an Irish secondary school curricular development initiative involving the phased development and introduction of new syllabuses in Mathematics. In particular we focus on the level of preparedness Project Maths offers students of an applied mathematical subject, namely Data Analysis for Decision Makers (DADM), at third level. We examine whether these students think that their experience of Strand 1 (Probability and Statistics) in the Leaving Certificate PM course affected their performance on their DADM module. INTRODUCTION Maths has been identified as a key driver for the knowledge economy, scientific advancement and as a fundamental requirement for active citizenship (Department of the Taoiseach, 2008). Project Maths (NCCA, 2013) is an initiative by the Irish education authorities which involves the introduction of revised mathematics syllabuses at second level. It involves changes to what students learn, how they learn it and how they are assessed. It aims to develop the mathematical knowledge, skills and understanding needed for continuing education, life and work. It also purports to develop a flexible, disciplined way of thinking in students to enable them to solve problems in mathematical and real world contexts. Similar initiatives are ongoing across the United States, (Weiss, Pasley, Smith, Banilower & Heck, 2003), and Europe (Eurydice, 2011) in countries such as Belgium, the Czech Republic, Slovenia and Spain, encouraging pupils to participate in their own learning through discussions, project work, practical exercises and other ways that help them reflect upon and explain their mathematics learning. The latter report (Eurydice, 2011, p.56) explores the common challenges and national policies for teaching mathematics in Europe. It found that the use of problem-focused learning is the main objective of many countries. Some countries emphasise contexts which are familiar to students so as to provide a meaningful frame of reference for their learning (Spain, Poland and Italy). In other countries (Estonia), students are encouraged to participate in outdoor learning, relating their mathematical knowledge to creativity, architecture and visual arts. Similarly, active learning and critical thinking is advocated in many jurisdictions (for example Belgium, the Czech Republic, Slovenia and Spain), encouraging “pupils to participate in their own learning through discussions, project work, practical exercises and other ways that help them reflect upon and explain their mathematics learning”. Indeed some of the central aims of the PM syllabus (Project maths Development Team, 2013) are: the application of mathematics to real-life settings; forming links between maths topics; using mathematical language and verbal reasoning to convey ideas; and planning and conducting collaborative investigative activities. Strand 1 of the revised syllabus in Ireland concerns statistics and probability. In 2011, the Quinn School of Business, UCD, Ireland restructured its undergraduate business degree programmes. During this process quantitative skills were identified as central to the holistic education of business students. Data Analysis for Decision Makers (DADM) was designed during this review as a core module for all (circa 600 per annum) undergraduate School of Business students. It is delivered in Semester 1 of first year. In this paper we aim to analyse undergraduate business student perceptions of Data Analysis themes and examine whether earlier exposure through Project Maths at second level, to similar themes assists students in meeting the DADM learning outcomes. To aid our understanding of the research we ask the following question in this article: Did exposure to probability and statistics concepts, via Project Maths, assist students meet their DADM learning outcomes? BACKGROUND MATERIAL An interim report on Project Maths (PM), based on research commissioned by the NCCA and conducted by the National Foundation for Educational Research in England (Jeffes, Jones, Cunningham, Dawson, Cooper, Straw, Sturman & O'Kane, 2012) looked at the pilot schools where Project Maths has been taught since 2008. This report also examined the rest of the country’s schools which have taught PM since 2010. It findings suggest: • Students at Leaving Certificate level appear to be performing well and are broadly confident in their abilities in many aspects, particularly Statistics and Probability. • Students find tasks that require higher order skills such as reasoning and transferring knowledge to new contexts harder than more mechanically demanding tasks. • Students are regularly engaging with the application of mathematics to real-life situations; making connections and links between mathematics topics; using mathematical language and verbal reasoning to convey ideas; and planning and conducting investigations. • Whilst, in general, students following the revised syllabus performed better than their comparison group peers, this difference is only statistically significant in relation to a particular item which explores students’ abilities in Strand 1, Statistics and Probability (assessing students’ understanding of the outcomes of simple random processes). • Many Leaving Certificate students were planning to pursue further study and/or careers in mathematics, favouring professions such as accountancy and business management. While we acknowledge that it will be 2016 before the first full cohort of first to sixth year (the secondary school cycle in Ireland) students will have taken Project Maths, we hope this paper will serve to promote a constructive discussion on the readiness of our second level mathematics students for a more contextualised data analytics course at third level. Apart from Jeffes et al. (2012), no empirical research on the impacts of Project Maths on student learning and achievement has yet been published. In this discussion we offer some comparative data on second and third level student achievement scores in the quantitative disciplines. A report by the NCCA (2012) of mathematics teachers in the 24 initial schools providing Project Maths surveys students’ experiences with PM. In this article we give an airing to some of the student’s attitudes towards PM and its relevance to the third level module DADM. UNDERGRADUATE DATA ANALYSIS The state end of secondary school exam in Ireland is the Leaving Certificate. A minimum of a B3 (70%-74%) at ordinary level or a D3 (40%-44%) at higher level in the Irish Leaving Certificate Maths is stipulated as an entry requirement for undergraduate business programmes at UCD. The majority of entrants to business programmes in UCD come via this access route, (89% in 2012). Students who sat the Leaving Certificate in 2012 were the first cohort to have come through the Project Maths Strand (Strand 1) on Probability and Statistics. Equivalent entry requirements are stipulated for other access routes. Business students come into the business programmes with a wide variety of mathematical abilities. Successful completion of the DADM module means that students are able to: 1. Prepare spreadsheet models to store, manipulate and analyse quantitative data using common probability distributions and statistical functions. 2. Calculate, analyse and present useful statistical measurements from large-scale data sets. 3. Create and interpret inferential statistical statements about population parameters. DADM Delivery Approach The module is delivered through a two hour lecture in large groups (circa 150) with a followon tutorial in smaller groups (circa 45). We use the principles of Active Learning (AL) in DADM contact time. AL can be defined as a process of keeping students mentally active in their learning through activities that involve them in gathering information, thinking and problem solving. The interested reader may consult Michael (2006) for evidence that this approach is effective. A business related problem with a small exercise is used during lectures as the basis for developing theoretical concepts. Students are then asked to engage in class exercises, usually with their peers, to answer similar problems and interpret their solutions. This approach is aligned well with the student’s experience of Project Maths at second level where discussion and collaborative learning are encouraged. DADM also employs eLearning and ICT resources extensively which students can access online at any time. The learning management system used in UCD is Blackboard, through which the eLearning content is made available. The eLearning authoring software Articulate Storyline (Articulate, 2013) is used to develop the eLearning content. Captivate (Adobe, 2013), is also used to record “How to” demonstrations in Excel. Students were also invited to complete Excel spreadsheet training given by an external IT training provider in weeks 3 and 4 of the semester. Students are instructed to complete additional online short exercises in advance of tutorials. Answers to the online exercises (and feedback on any calculation steps) are given when a student submits an attempt. The short exercises focus on methodology and demonstrate the steps in formulating and solving problems. Students can attempt these exercises as many times as they like. We encourage students to use these as practice exercises. There is no continuous assessment (CA) credit awarded for attempting these online exercises. At the tutorial, tutors review student progress with the online exercises, responding to student queries and clarifying concepts. Online review and more extensive case study exercises are available for students to work through in their own time at the end of each section of the module. These exercises, called Theory into Practice (TIP) focus on sample business applications and demonstrate the applicability of data analysis within a business context. Sample analysis and interpretation are given when a student submits an attempt. Open source data sets are used where possible. Again sample analysis and interpretations are given as feedback when a student submits an attempt. Our intention is to facilitate different student learning styles and abilities so as to enable students to engage fully with the learning process. We try to engage students in ways of learning which students prefer while maintaining academic rigour with respect to the content. While there is much debate currently as to whether CA should be introduced on the Leaving Certificate programme the approach taken on the DADM module is to allocate 40% of credit for CA and this leads to a very different learning environment to the secondary school classroom. PROJECT MATHS – STRAND 1: STATISTICS AND PROBABILITY Concern that the teaching and learning of mathematics in secondary schools in Ireland was failing to make the connections between mathematics and its place in real life, led to a curriculum review and the roll out of the Project Maths initiative. An in-depth study of the Irish mathematical classroom (Lyons, Lynch, Close, Sheerin & Boland, 2003) showed that most of the learning taking place in the maths classroom was of a procedural and mechanical style and that the teaching and learning approaches promoted this learning. Indeed Irelands relatively poor showing in international comparison tests such as PISA in 2003 (Ireland ranked 17th of 29 OECD countries and 20th of the 40 participating countries) and TIMSS in 2011 (Ireland placed 17th again here whereas Ireland was placed significantly above the international mean and was ranked 6th of 17 countries when it last took part in TIMSS in 1995) also prompted these major syllabus changes. While the rationale behind Project Maths has been widely welcomed by the key educational stakeholders, its execution has drawn mixed reaction and often heated debate. Oldham & Close (2009) argue that more in-service teaching must be provided for the teachers to become more comfortable with a style of teaching that may not be what they themselves are used to or even contrary to their own philosophy of mathematics education. A major professional development initiative (The Professional Diploma in Mathematics for Teaching) designed in UL and NUIG, to up skill the large cohort (48%) of out-of -field secondary maths teachers is now underway across 10 Irish third level institutions including UCD. The PM syllabus and its assessment has also attracted attention. For example Engineers Ireland (2010) and some third level educators (Grannell, Barry, Cronin, Holland & Hurley, 2011) have expressed concern at the omission of key maths topics such as integration, matrices and vectors and the over influence of the PISA type mathematical philosophy. Conversely the topics of statistics and probability have been given twice as much coverage within the syllabus compared to previous syllabuses (Lubienski, 2011, p. 31) has noted the length and difficulty of the statistics strand, particularly for senior cycle students) and indications (Jeffes et al., 2012, p. 8) show that these topics are being learned well in line with the PM objectives upon international comparison. There is considerable overlap between the learning outcomes of Project Maths Strand 1 (Probability and Statistics) and the DADM module. Learning outcomes for Strand 1 specify that students should be able to: • Discuss basic probability rules and concepts. • Calculate probabilities associated with the outcomes of random processes. • Use statistical reasoning with the aim of becoming a statistically aware consumer. • Discuss and design plans for data gathering. • Describe data by means of descriptive statistical and graphical techniques. • Draw inferences from data. • Synthesise and solve related problems. Traditionally, there has been a trend to ‘teach to the test’ in leaving certificate subjects such as maths. There is an argument that the introduction of coursework would alleviate that situation and facilitate a deeper knowledge and promote exploration of subject content. Problem Based Learning (PBL) and Enquiry Based Learning (EBL) are effective teaching strategies but may be more appropriate for adult or distance learners and third level environments. Lubienski (2011) notes the high emphasis on the Leaving Certificate examination, which, in her view, constrains instruction and places the second level instructor in the role of “exam coach”. Initial feedback suggests that teachers feel under pressure to revert to old style ‘drill and practice’ teaching and abandon student-centred, inquiry-based methodologies because the Leaving Certificate result is so important in gaining a place at college (Cosgrove, Perkins, Shiel, Fish & McGuinness, 2012). An investigation by the authors of how Continuous Assessment may support learning at second level will be the topic of a related discussion to the one presented here. Assessment Strategies To enable the students to meet the specified learning outcomes and achieve the goal of deeper learning we endeavour to use a variety of assessment methods. Grading criteria are employed to quantify to what extent individual students have met the learning outcomes. The DADM assessment tasks are summarised in Table 1. Item Weight (%) Comment Excel work 3 Spreadsheet exercises Excel test 12 Data analysis 2x MCQs 15 Theory & practice Team project 10 Open ended Terminal exam 60 2 hr. written exam Table 1: DADM assessment RESEARCH METHODOLOGY We obtained ethical clearance to conduct our action research project. Both quantitative and qualitative data were gathered in an attempt to answer the research question stated in Section 1. We compared the students Leaving Certificate results from 2012 with their performance on the DADM module 2012. We also implemented an optional online survey to ascertain student’s attitudes toward the DADM module and their experiences with Project Maths. We acknowledge the limitations of web-based surveys and also accept that the use of an optional online survey is limited in use for a scientific study and we refer the reader to Couper (2000), Dillman & Bowker (2001), and Umbach (2004) for a discussion on the problems arising from online surveys. Considering the timeframe constraints of the academic semester this was deemed by the authors to be the only approach available to elicit student opinion and was judged to be useful in giving some insight into the research question under discussion, notwithstanding the issue of a self-selecting response mechanism. The survey contained a mix of five-point Likert items and open ended questions. The Likert items were scaled from ‘Strongly Disagree’= 1 to ‘Strongly Agree’= 5. Using this scale, the more students agree with a statement, the higher the Likert average. A justification of the appropriateness of Likert scales to gauge attitudes is given in Waples, Weyhrauch, Connell & Culbertson (2010). A copy of the survey and anonymised data are available at the author’s home page http://www.ucd.ie/cba/members/paulacarroll/ The survey was delivered via Blackboard. A pilot 'pen and paper' survey was run in week 8 of semester following which, minor changes were made to improve readability. The full survey was made available to students from week 11. Of the 546 students enrolled for the module, a small number withdrew from their programme of study by the end of semester 1 for various personal and academic reasons. This left 540 students of whom 127 responded to our invitation to participate in the survey giving a response rate of almost 24%. We acknowledge that in social scientific terms this is a low response rate and note also that not all respondents answered all the survey questions. Students are also invited to complete an end of semester evaluation for each module they complete through UCD’s centralised evaluation system. For the DADM module 135 students completed this evaluation. Our survey response rate is in line with similar evaluation surveys conducted by the university. ANALYSIS AND FINDINGS Question 10 and 11 respectively, of the survey stated: On completing (Having completed) DADM (Project Maths) I will have (I have) a good understanding of Probability and Statistics. Table 2 shows a summary of the responses to questions 10 and 11. It shows the percentage of responses in each Likert category for each course. The last two rows show the Likert mean and standard deviation excluding students who did not answer that question. We see for example that 41% (10.2%+30.7%) of respondents did not agree to having a good understanding of Probability and Statistics on completing DADM. While 12.6% of respondents said they strongly disagreed that they had a good understanding of the topics having completed PM. DADM Project Maths Unanswered % 12.6 11.8 Strongly Disagree % 10.2 12.6 Disagree % 30.7 16.5 Neither Agree nor Disagree % 18.1 13.3 25.2 32.2 Agree % 3.1 1.6 0 11.8 Likert mean* 2.77 2.91 Likert Stdev* 1.1 1.1 Strongly Agree % NA % Table 2: Student attitudes to DADM/PM levels of understanding (*excluding non-response) As stated earlier the learning outcomes for PM (Strand 1) and DADM are similar but the results above would suggest that the DADM respondents are struggling with the applications and context to business of the probability and statistics learning material. This is a worrying development as an analysis of the Project Maths text books (Morris, Cooke & Behan, 2012; Keating, Mulvany, Murphy & O’Loughlin, 2012), and the State Examinations Commission (2013) show a heavy emphasis on business type applications and financial contexts/scenarios in relation to questions on statistics, data and probability. We analysed the DADM results of students who completed the module and whose access method was the Leaving Certificate in 2012 (this accounted for 429 of the entire DADM class, 179 females and 250 males). The left side of Figure 1 shows a histogram of the leaving cert points in maths (excluding the bonus points for higher level maths). A bimodal distribution is apparent. The right side of Figure 1 shows the DADM equivalent grade points for the same cohort of students. The distribution is less bimodal, leaning toward a right skewed unimodal distribution. The Pearson correlation coefficient measuring the relationship between the LC 2012 and DADM results is 0.498. At p = 0.05, using a t test, we conclude that there is evidence of correlation although it could be considered to be relatively weak. There is evidence to suggest that the more capable students (those who achieved an honour in the higher level leaving certificate) may have become complacent in thinking that Project Maths was very similar to DADM and as such didn't do the necessary work to attain the same high results they achieved at second level. While the students who took the ordinary level course (or those who had gotten a D at higher level or those from GCSE entry route) worked harder. In fact those who studied maths at the ordinary level (126 students) scored an average of 47.73% overall on the DADM module while the higher level people (303 students) scored an average of just 57.39% with standard deviations of 12.09% and 12.06% respectively. Figure 1: Leaving Cert 2012 Maths points (left) and DADM grades (right) One student commented that “the methods through which we learned probability and statistics for project maths and this module are different”. In terms of the final assessment, the Concepts and Skills section of the PM Leaving Certificate higher level paper (paper two) mirrors the short questions section of the DADM terminal exam paper, both in terms of content and style. The Contexts and Applications section of the PM paper is again closely aligned to the long questions (applied) on the DADM paper. The average mark scored on the long questions (more applied and inferential, PM-type questions) was just 39% while the short (more methodical and formulaic) questions scored an average of 56%. A comparison of the word count for the longer sections of both papers shows no significant statistical difference. The sample papers for 2011, 2012 and 2013 combined with the actual 2012 PM paper had an average word count of 486 (there were 598 words on the 2012 paper taken by the respondents in question). The word count for the 2011 and 2012 DADM papers averaged at 422 with the 2012 paper containing 429 words. Considering that the Leaving Cert paper is 2.5 hours in duration compared with 2 hours for the DADM exam, these word counts are notably similar. The word count for the Maths for Business module (the other numerical module taken by these students) exam papers (also 2 hours in length) for the last 3 years was 728 words. Note that this is the word count for the entire exam as there is no differentiation between short and long questions here. The IMTA (Irish Maths Teachers Association) said the wordy nature of some questions [on the Leaving Cert paper] has posed problems for students with literacy difficulties. In relation to Project Maths one student said: “The maths became too English based and it was difficult to figure out at times what the actual sum was”; another said: I find statistics and probability difficult anyway, and studying Project Maths didn’t help me in any way shape or form. I found it difficult enough to understand and interpret the amount of writing and English that came with the questions before I was even able to begin understanding and attempting the maths side of it. I don’t find the way that Project Maths was taught to be effective, especially when the teachers didn’t even seem to fully understand it. Students of the DADM module expressed a preference for concrete tasks like calculating a measure in Excel and found higher level synthesis tasks more challenging. Students also found the language and terminology difficult to grasp. One student commented: Some of the slides on Blackboard were helpful but alot them (sic) used very complicated language which meant that I was at a loss to what many of them were actually explaining. Preliminary analysis of the Maths for Business module, shows an uplift in results compared to previous years, which is not apparent in DADM. This is likely due to the increased number of students with higher level maths due to the bonus points initiative. The fact that there is an emphasis in DADM on the higher level of learning/understanding required to synthesise problems may also be reflected in this analysis. Two opposing views stated by students were: “the Project Maths at pass level did not prepare me for the level of maths in Data analysis [DADM].”, “DADM is a very realistic topic which can be applied in real life problems and maybe should be focused on more in project maths.” To determine second level students’ views on the broader application of mathematics, in (4) they were asked to comment on the extent to which they perceived it to be useful in the following ways: • to help in daily life • to aid learning in other school subjects • to enable them to get into the university of their choice • to enable them to get the job of their choice. 80-100% of these students said mathematics as a career was relevant to professions such as • Accountant • Engineer • Owning a business • Scientist • Working with technology • Sales assistant Students of DADM were asked to reflect on their experiences with DADM and with Project Maths, and to indicate if they thought the skills and understanding developed were relevant to consumers, business leaders, scientists or engineers and government or public bodies policy makers. Their responses are summarised in Table 3. Category Considered relevant Unanswered 14.17% Consumer 22.05% Business leader 66.14% Scientist or engineer 44.09% Government policy maker 33.86% Table 3: Student perception of the relevance of DADM skills The business leader category was selected by 66.14% of survey respondents as having relevance for the DADM module. Interestingly only 22.5% thought the DADM skills relevant to a consumer. CONCLUSIONS Over 84% of students registered to DADM achieved a sufficient level of understanding as indicated by the pass ratio. There was a relatively weak correlation between exam scores on the 2012 Leaving Certificate mathematics and the semester 1 2012/13 DADM module. The performance of the higher level Leaving Certificate was not as high as expected when compared to their ordinary level counter parts. Analysis of the more procedural questions versus the context-based question on the DADM exam resonates with the finding by Jeffes et al. (2012) which concluded in general, items requiring higher order skills (such as reasoning and an ability to transfer knowledge to new contexts) are found more difficult than those which are more mechanical in demand. The readiness of PM students to study a more contextualised data analytics course at third level is unclear and perhaps it is too early in the development of the PM initiative to draw any significant conclusions. It is interesting in this MOOC saturated era that 55% of students think that lectures do not help them learn how to implement data analysis techniques and interpret the results. If the university experience is about promoting self-directed learning we should not be surprised that first year students find this a discomforting experience for which they do not thank us. As reported in Jeffes et al. (2012), the proposed teaching and learning approaches promoted by PM are being felt by the students but we cannot yet tell if the students new mathematical experiences are as a result of the content and delivery of PM in terms of the teaching they receive or if their own learning style is in fact changing. As Project Maths is as much about changing teaching and learning practices as it is about changing content, third level instructors also must take on board the two-way process that is PM and adapt to the changes taking place in second level mathematics teaching and learning. We suggest that this paper be viewed, not as a critique of whether PM is working or not, but as a wider commentary on the educational aspirations of improved literacy and numeracy for our students and how we at third level can contribute positively to empowering this. ACKNOWLEDGMENTS We thank the DADM students for participating in this research project. We also thank the Quinn School of Business eLearning and programme office teams for their support. REFERENCES Adobe (2013). Adobe Captivate. Retrieved 8/8/2013 http://www.adobe.com/ie/products/captivate.html Articulate (2013). Articulate storyline. Retrieved 8/8/2013 http://www.articulate.com/ Cosgrove, J., Perkins, R., Shiel, G. Fish, R. & McGuinness, L. (2012). Teaching and Learning in Project Maths: Insights from Teachers who Participated in PISA 2012. Dublin: Educational Research Centre. Retrieved 8/8/2013 http://www.ncca.ie/en/Conference/Teaching_and_Learning_in_Project_Maths_Web_Repo rt.pdf Couper, M.P. (2000). Usability evaluation of computer-assisted survey instruments. Social Science Computer Review, 18(4), 384-396. Department of the Taoiseach (2008). Building Ireland’s smart economy: A framework for sustainable economic renewal. Dublin: The Stationery Office. Retrieved 8/8/2013 http://www.taoiseach.gov.ie/attached_files/BuildingIrelandsSmartEconomy.pdf Dillman, D. A. & Bowker D. (2001). The web questionnaire challenge to survey methodologists. In U. -D. Reips and M. Bosnjak (Eds.), Dimensions of Internet Science (pp. 159-178). Lengerich: Pabst Science Publishers. Engineers Ireland (2010). Report of the Task Force on Education of Mathematics and Science at Second Level. Retrieved 8/8/2013 http://www.engineersireland.ie/public/20100211mathematics_and_science_at_second_level.pdf Eurydice (2011). Mathematics Education in Europe: Common Challenges and National Policies. Retrieved 8/8/2013 http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/132en.pdf Grannell J., Barry P.D., Cronin M., Holland F. & Hurley D. (2011). Interim Report on Project Maths. Cork: School of Mathematical Sciences UCC. Retrieved 8/8/2013 http://143.239.128.67/en/euclid/edu_and_careers/projectmaths/ Jeffes J., Jones E., Cunningham R., Dawson A., Cooper L., Straw S., Sturman L. & O'Kane M. (2012). Research into the impact of Project Maths on student achievement, learning and motivation. Slough: National Foundation for Educational Research. Retrieved 8/8/2013 http://www.ncca.ie/en/Conference/Impact%20of%20Project%20Maths%20Final.pdf Keating M., Mulvany D., Murphy, O. & O'Loughlin J. (2012). Active Maths 4. Dublin: Folens. Lubienski, S. (2011). Mathematics Education and Reform in Ireland: An Outsiders Analysis of Project Maths. Bulletin of the Irish Mathematical Society, 67, 27-55. Lyons, M., Lynch, K., Close, S., Sheerin, E. & Boland, P. (2003). Inside classrooms: The teaching and learning of mathematics in social context. Dublin: IPA. Michael, J. (2006). Where’s the evidence that active learning works? Adv Physiol Educ, 30, 159-167. Morris O.D., Cooke P. & Behan P. (2012). Texts and Tests 4. Dublin: The Celtic Press. NCCA (2012). Project Maths: Reviewing the project in the initial group of 24 schools – A report on school visits. Retrieved 8/8/2013 http://www.ncca.ie/en/Curriculum_and_Assessment/PostPrimary_Education/Project_Maths/Resources/Experiences_of_teachers.pdf NCCA (2013). Retrieved 8/8/2013 http://www.ncca.ie/en/Curriculum_and_Assessment/PostPrimary_Education/Project_Maths/ Oldham E. & Close S. (2009). Solving problems in mathematics education: Challenges for Project Maths. In D. Corcoran, T. Dooley, S. Close and R. Ward (Eds.), Proceedings of Third National Conference on Research in Mathematics Education. (pp. 296-309). Dublin: St. Patrick’s College. Project Maths Development Team (2013). Retrieved 8/8/2013 http://www.projectmaths.ie/overview State Examinations Commission (2013). Retrieved 8/8/2013 http://www.examinations.ie Umbach, P. D. (2004). Web surveys: Best practices. New Directions for Institutional Research, 121, 23-38. Waples, C., Weyhrauch W., Connell A. & Culbertson S. (2010). Questionable defeats and discounted victories for Likert rating scales. Industrial & Organizational Psychology, 3(4), 477-480. Weiss I.R., Pasley J.D., Smith P.S., Banilower E. & Heck D. (2003). Looking Inside the Classroom: A Study of K-12 Mathematics and Science Education in the United States. Chapel Hill, NC: Horizon Research, Inc. Retrieved 8/8/2013 http://www.horizonresearch.com/insidetheclassroom/reports/looking/complete.pdf