Observation

Observation
Graham Tall research@grahamtall.com September 2003

The emphasis here is on structured observation: observing individuals in carefully defined situations. The much broader socio-anthropological concept of of unstructured observation which in some instances is associated with ethnographic ‘Participant observation’ is not considered here.

I. Scientific (systematic/structured) Observation

Just as with questionnaires and interviews there is a ‘scientific’ as well as a ‘socio-anthropological’ approach to observation. If anything, however, the differences are even more polarised. The explicit assumption, in scientific observation, is that the range of behaviour being observed is either already known or will be identified by careful analysis:

The OPTIC research of Merrett and Wheldall (1986) and Formal Observations used by Mike Tobin requires the researcher to define exactly what is being studied so that the frequency of the event can be accurately monitored.

Well known curriculum observation schedules like FIAC (Flanders Interaction Analysis Categories - Flander (1970)) or Egglestone’s (1975) ‘Science Teaching Observation Schedule’ work by requiring the researcher to categorise what is happening at regular intervals - Flander’s schedule requires that a note should be made every 3 seconds, Eggleston required observers to record continually, moving to a new column every 3 minutes.

Just as with questionnaires and interviews the observation tool must fit the actual teaching process being studied; it is self-evident that FIAC and Egglestone’s science observation schedules are measuring something different.

"A study of Flander’s ten categories demonstrates clearly that the categories relate to traditional classroom teaching. The categories would be irrelevant for assessing an active learning situation let alone the science teaching procedure used by the teacher" (Egglestone (1975)).

But, as Egglestone demonstrated, acarefully designed schedules can accurately identify other styles of teaching. Hence, whilst the concept of a structured observation schedule can be used across a range of teaching situations; the schedule has to be relevant, analysing the frequency of the specific behaviours under study.

The strength of such schedules is that they allow the researcher to identify the frequency or order of events and encourage comparisons over time. They can therefore be used to:

a)	Obtain an overview of how large numbers of teachers teach - typical of survey type research. Eggleston et.al.
b)	Accurately monitor the frequency of specific aspects of teacher behaviour, so allowing detailed comparisons over a period of time. e.g. behavioural analysis OPTIC schedule designed by Merrett and Wheldall.

When used in monitoring and/or appraisal, the strength of observation schedules, is that the observer and the teacher can agree in advance what is being studied and the observer can make an accurate record for future comparison.

The weaknesses of observation schedules are that they do not record the contextual data concerning the students being taught, qualitative impressions such as ‘was the lesson interesting’. In monitoring or appraisal it is demonstrably necessary to recognise the danger of over-simple classifications:

"to describe... fish that are caught in terms of their length and weight is to reduce radically what we can know about the qualitative features of the ones that have been caught, not to mention the features of those that the net failed to catch in the first place." (Eisner, 1979)

II. An Example of a Simple Observation Approach

The observation discussed below is described in the following thesis and article:

Tall, G.E. (1999) The Scientific Paradigm’s Relevance to Curriculum Studies: The Experiences of an Individual
Researcher. University of Birmingham, Unpublished Ph.D. Thesis
Tall, G. & Hurman, J. (1999) Using a Dictionary in A French Examination: The Students’ Experience, Language
Learning Journal Issue 20 or 21 Publication agreed

a) Context
The Students sat the papers under examination conditions.

Object of Exercise
1. To obtain an objective record of the extent of dictionary use.
2. To assess the reliability of students own record of circling words or phrases when using a dictionary.
Reliability is demonstrated if the number of circles correlates reasonably well with the objective record of the observer.

Definition of Dictionary Use
It is insufficient for a student to simply be holding a dictionary or aimlessly flicking through the pages; it is essential to judge dictionary use by the following criteria, one of the following:

1. Turning the dictionary pages as in a search
2. Running a finger down a page
3. Reading the dictionary
4. Copying from the dictionary
5. Referring between dictionary & examination paper

Insufficient Criteria on their own:
1. Dictionary open in front of student
2. Dictionary held by student
3. Apparent random flicking through the pages
4. Putting the dictionary down If criteria 1, 2 or 3 are observed, watch the student a little longer to avoid mis-diagnosis
If, with criterion 4, the student wasn’t identified in the preceding observation, hash the box.

b) Observation Recording Schedule Used: 16 Students sitting together:

Quantitative Observation and Recording:
1. Each student is represented by a box divided into quadrants
2. Write the student’s number in the shaded box
3. FOR ONE MINUTE IN EVERY TWO, FILL IN A RECTANGLE IF THE PUPIL USES
        A DICTIONARY. LEAVE RECTANGLE BLANK IF DICTIONARY IS NOT USED.
        a. For the 1st. 15 minutes colour/leave blank the rectangles in quadrant 1
       b. For the 2nd. 15 minutes colour/leave blank the rectangles in quadrant 2
       c. For the 3rd. 15 minutes colour/leave blank the rectangles in quadrant 3
       d. For the 4th. 15 minutes colour/leave blank the rectangles in quadrant 4

EXPLANATION:

Observation schedule sheet:

c) Summary Findings Concerning the Use of an Modern Language Dictionary
in GCSE Examinations

Three hundred and one candidates were observed in 16 schools. At the Higher Tier, of the 30 observations possible in the 60 minute examination, a dictionary was used on average just under 15 times. A similar result was observed with Foundation Tier candidates indicating that candidates generally used the dictionary every other minute.

In order to make compare Foundation and Higher candidates use of the dictionary, the number of observations was standardised to a maximum of seven for each of the following periods of time:

(a) the first Ľ hour,

(b) the last Ľ hour

and

Candidates were then classified according to whether their number of dictionary uses in any one period was greater than two thirds of the combined number of their uses in the other two periods. The column headed ‘Not greater’ (Table 1) includes 16% of candidates who tended to use the dictionary continually through the examination.

The figures in the ‘Last’ column in table 1 indicate that relatively few Foundation or Higher Tier candidates primarily used their dictionary as a final check at the end of an examination. Foundation candidates appeared to use their dictionary to understand the questions whilst Higher Tier candidates appeared to use it in the middle part of the exam to write their answers.

Table 1: Dictionary Use by Individual Candidates by Period of Time

		Greater than other 2 periods of time			NotGreater
	Period of time	First	Middle	Last	Other	Total
Foundation Tier	Number of Candidates	59	44	15	29	147
	%	40%	30%	10%	20%
Higher Tier	Number of Candidates	47	70	18	19	154
	%	30%	45%	12%	13%
Both Tiers	Number of Candidates	106	114	33	48	301
	%	35%	38%	11%	16%

REFERENCES:

Egglestone, J.F., et al. (1975) A Science Teaching Observation Schedule. Macmillan Education
Flander, N.A. (1970) Analysing Teacher Behaviour, Reading, Mass. Addison Wesley
Good T.L., Brophy J.E., (1984) Looking in Classrooms (3rd. Edition) Harper Row
Irwin, D.M. & Bushnell, M.M. (1980) Observational Strategies for Child Study. Holt, Rhinehart and Winston:London
Wragg, E., et al. (        ) Education Studies: A Second Level Open University Course. Personality and Learning.
            E201 Block 11, Classroom Interaction. (Describes Flanders Observation Schedule)
Merrett, F. and Wheldall, K. (1986) Observing Pupils and Teachers in Classrooms (OPTIC): a behavioural observation
            schedule for use in schools. Educational Psychology 6, 57-70

Home Page Research Introduction Quantitative Advice Index Statistical Tests Research and Statistics Courses