Oral Communications Assessment Report – SLO f

Written by:  Diane Schwartz

 

6/22/11

 

1.               Student Learning Outcome f: An ability to communicate effectively with a range of audiences- Oral Communications

 

2.               Method(s) of Assessment:  

( Assessment #1) Two faculty members (Diane Schwartz and Robert Lingard) attended and evaluated 16 oral presentations made by students in the Comp 450 (Computers and Society) and Comp 491L (Senior Project Design Lab) in May 2011. The students in Comp 450 made 10 – 15 minutes oral presentations on issues relating to societal impacts of computing. The students in Comp 491L gave 30 minute oral presentations of their year-long senior design project. The presentations in Comp 491L were team presentation where each student made a part of the team project presentation. Each student spoke for about 5 -8 minutes. Each student presentation was assessed by both of the faculty members using a rubric and evaluation instrument developed by a College and Department faculty.  The evaluation instrument was developed using the key performance indicators for this student learning outcome. The instrument was used to evaluate the student presentations on eleven oral communication standards, with possible scores from 1 to 4 on each standard. The interpretation of scores 1 to 4 are 1 = Weak; 2 = Adequate; 3 = Good; 4 = Excellent. (The evaluation instrument is in Appendix A. The Key Performance Indicators are in Appendix E.)

 

(Assessment #2) We also assessed the results of the CECS Senior Design Showcase Competition held in April 2011.  Three teams of computer science seniors presented their senior design project work or other projects to a panel of two judges from industry who rated the students on their oral presentations and on the general quality of their project. The judges used a rubric and scoring sheet developed by the College. (The scoring sheet is in Appendix C).

 

 

3.               Results of the Assessment #1:  The results of Assessment #1 showed that 81% (13/16) of the students assessed have adequate to excellent oral communication skills on each standard measured. The students, as a whole, performed best on standards 4, 10 and 11 of the evaluation instrument. Standards 4, 10, 11 deal with their ability to communicate the intended information in a manner that was focused, appropriate for the intended audience and professional. The three students who were rated “Weak” on at least one standard exhibited weaknesses in standards1, 2, 6, and 8. Standards 1, 2, 6 and 8 measure the quality of the visual aids; an effective speaking style; the organization of the presentation; and whether or not the presentation met the requirements of the assignment. (See attached charts in Appendix B for details of the results)

 

 

4.               Results of Assessment #2:  Three teams from Computer Science gave presentations at the CECS Senior Design Showcase. The industry judges gave the student teams total scores of 82 to 90 out of 100 possible points. Using the judging rubrics we interpret this to translate to “good” oral presentations. In the individual scoring categories the lowest team scores were in category “clarity of presentation” (Mean = 15.6 / 20 points).  The highest scores in project methodology and project completeness (Respective means = 18.5/20 and 13.8/15).  (See attached charts in Appendix D for details).

 

 

5.               Analysis of the Assessment Results. 

( Assessment #1)The students in the senior design project class (Comp 491L) performed well and all satisfied the key performance indicators of student learning outcome for oral communication (SLO f).   The students in the Computers and Society class (Comp 450) did not all perform as well. Three of the eight students gave weak oral presentations, exhibiting poor oral presentation skills and lack of adherence to the requirements of the assignment. The remaining five all satisfied the key performance indicators of student learning outcome for oral communications (SLO f)

 

(Assessment #2) The student teams performed adequately in the Senior Design Showcase. Two of the three teams scored relatively low (13.5/20 and 15/20) on the “clarity of presentation” measure. This measure rated the content of their presentation and required the students to clearly explain their project objectives and the logical development of their project.

 

 

6.               Recommendations for Actions/Changes: 

 

a.     If this was an informal assessment, is there a need to perform formal assessment(s) with respect to this SLO?

This was a formal assessment.

 

b.     If this was a formal assessment, should it be repeated?  If so, when?

Yes, we should repeat this assessment every three years. The next assessment should be in Spring 2014.

 

c.     Should changes be made in the way this assessment was done?  If so, describe the changes.

 

(1)   We should try to have two faculty evaluators evaluate all of the presentations in Comp 491L and Comp 490.  ( Comp 490 will replace Comp 450 by Spring 2014 and so the evaluation will take place in Comp 490)

 

(2)   Students should be given the oral assessment evaluation criteria at least a week before their presentation. The Comp 491L students were given the criteria before the evaluation. They were not given to the Comp 450 students.

 

(3)   Hard copies of the student power point presentations should be collected before the presentation so that the evaluators can make notes on the presentation hard copy.

 

(4)   All student teams in our senior design course should participate in the Senior Design Showcase, both the oral presentation and the project display.  This will give students an opportunity to speak in front of an industry audience.

 

 

d.     Should there be any changes in curriculum based on the results of this assessment?  If so, describe recommended changes. 

 

All students should make three or four 15-minute oral presentations by before they take their senior design courses (Comp 490/L & 491L) and then make at least two oral presentations in senior design.  Using the oral communication rubrics developed by the Department and College, the faculty should review with the students what they need to do to make a professional quality presentation.

 

e.     Should any other changes be made?   No other changes are recommended at this time.

 

Appendix A:    Assessment # 1 

Assessment of Comp 491 and Comp 450

 

Computer Science Dept                                                Spring 2011

 

 

Oral Presentation Evaluation

 

Course:  ____________________

 

Speaker __________________________________                            Date _______________

 

Title of Presentation __________________________________________

 

Evaluator _________________________________

 

 

Rating Criteria [See Rubrics]:   Excellent (4);   Good (3);   Adequate (2); Weak (1)

 

ANALYSIS/DISCUSSION/ORGANIZATION                                                        

  1. Presentation completely addressed topic and requirements of assignment.

 

  4           3             2             1
  1. Presentation provided appropriate level of analysis, discussion and evaluation as required by assignment.

  4           3             2             1
  1. Presentation was well organized in overall structure, beginning with a clear statement of problem and ending with a clear conclusion.

  4           3             2             1
  1. Presentation was focused and appropriate for the intended audience.

  4           3             2             1

 

STYLE/FORM AND FORMAT                                                  

  1. Visual aids were informative and helpful in understanding presentation.

  4           3             2             1
  1. The visual aids consisted of well-chosen words and graphics which complemented the speaker and were consistent with time limits of presentation.

  4           3             2             1
  1. Visual aids were clear and easy to read

  4           3             2             1

 

 

SPEAKING SKILLS / PROFESSIONALISM                                                                          

  1. Speaker used an effective speaking style.

  4           3             2             1

 
  1. Speaker engaged the audience’s attention.

  4           3             2             1

 
  1. Speaker communicated the intended information.

  4           3             2             1

 
  1. Speaker dressed appropriately for the audience and acted in a professional manner.

  4           3             2             1

 

 

 

COMMENTS:

           


 

APPENDIX B             DETAILS OF RESULTS OF ASSESSMENT # 1

 

 

 

 

 

 

 

 

 

APPENDIX C      Assessment # 2

Senior Design Showcase Competition Score Sheet

 

Senior Design Oral Presentation Scoring Sheet                                            April 2011

College of Engineering and Computer Science

California State University, Northridge

 

Project Title:

Department/Program:

Judge: 

 

Judging Categories/Rating Scale

Attributes

Score

 

Points for Clarity

0-9  :     Unclear

10-20:   Clear

 

 

• Design Objectives and specifics are stated clearly
• Logical Development
• Use of visual aids
• Use of examples and analogies

 

Points for Methodology

0-9: Questionable methods, dispersed activities, questionable data

 

10-20:  Appropriate and effective tools, compliance with design specification

• Utilization of current technologies
• Compliance with project specifications
• Originality of design, experimental procedures, processes or   construction

 

 Points for Interpretation of Results

0-7: Findings unclear, outcomes not met, weak interpretation of results

 

8-15:  Outcomes achieved. Well stated intriguing interpretations and appropriate extensions for further improvement

• Apparent technical and factual accuracy and grasp of the subject
• Did the team clearly present the necessary background, approach, results, and recommendations?

 

Points for Completeness of the project

 

0-7: Project is incomplete; poor teamwork

 

8-15: Project is complete. Significant teamwork.

• Is the project complete and has the team met or exceeded the desired design objectives?
• Is there evidence of the involvement of all team members in the project?

 

Points for Presenter’s ability to articulate significant project objectives and outcomes

 

0-4: Unremarkable, incomprehensible

 

5-10:  Well delivered. Informative and comprehensible.

• Organization: Has Introduction, body, and conclusions, with transitions between each
• Grammar, Fluency, and Choice of Words

 

Points for presenter’s ability to handle questions from the judges and the audience

 

0-9: Questions put off, bluffing, repetitions of assertions

 

10-20:  Excited by questions, answers provided new directions.

• Logical Development
• Clarity and Directness in Exposition
• Discussion
• Adherence to time

 

 

There are 100 possible total points.

 

APPENDIX D   DETAILS OF THE RESULTS OF ASSESSMENT #2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX E                   KEY PERFORMANCE INDICATORS FOR OUTCOME F

 

 

Outcome f: An ability to communicate effectively with a range of audiences

 

 

Key Performance Indicators for Oral Communication

1.     Student is able to give a well-organized oral presentation that completely addresses the topics and requirements of the assignment and is appropriate to the intended audience.

2.     Student uses an effective speaking style which generates interest in the audience and communicates intended information.

3.     Student is able to use visual aids which are informative and helpful to understanding the presentation.

4.     Student is able to give an effective oral presentation to an audience outside the University (e.g., to an industry or community audience).

 

Key Performance Indicators for Written Communication

1.     Student is able to write a well-organized term paper or project report that completely addresses the topics and requirements of the assignment.

2.     Student is able to provide support for main points of the paper with reasons, explanation and examples that are appropriate for the intended audience.

3.     Student is able to write papers that are correct in terms of syntax, grammar, spelling and required format.