GEOS 597e Spatiotemporal Data Analysis Workshop Home Page

GEOS 597e
Spatiotemporal Data Analysis Workshop

Fall 2006 offering. Questions? Email the instructor.

last updated December 6, 2006. Check back for updates frequently, and be sure to reload cached pages.
Note: as of December 11, 2004: files formally found at /xdisk/mnevans/stda/ on research.u.arizona.edu are now here.


Logistics	Overview	Acknowledgements	Help	Structure	Feedback	Syllabus/Schedule

Logistics

Units: 3
Meeting time and place: Wednesday, 1-4:50 pm, Computer Laboratory for Undergraduate Education (CLUE) Lab, Gould-Simpson Bldg., Rm. 228A.
Instructor: Michael Evans, ph (520) 626-2897. More about the instructor is here.
Office hours and location: during class time; also Mondays 2-3pm at the CLUE Lab, by appointment, and by email.

Reading and Course Materials: Readings from selected topical, authoritative and/or timely sources will be linked in the Schedule/Syllabus below as password-protected Portable Document Format (pdf) files. Lecture (preview) notes (may/may not include work and exercises performed in class) will be linked as html files and also as downloadable Microsoft PowerPoint-compatible files or pdfs. They may, but not guaranteeably so, be available prior to class meetings, and are subject to change and correction. We will make use of copyrighted materials under academic Fair Use Policy of applicable copyright law (see UA and U Texas notes on this, and what it means for you and us.)

Assignments: weekly Readings, pre-session assignment (Prework), and problem sets to be started in class and completed as homework (Classwork). Written and oral presentation of a final project applying the concepts and tools studied in this workshop is required. Look for Reading and Prework to be posted by Thursday 5pm the week before it is discussed; Classwork to be posted the day we attack it in class. If you need to work ahead of time due to prior commitments, please let me know well ahead of time so I can try to accomodate you.

Disability resources: Do you have a physical or learning disability? Please speak with me right away about accommodations. You will only need to have the Disability Resource Center send me a confidential documentation letter, and we can make arrangements from there.

Overview

How are the principal features buried in noisy spatiotemporal datasets routinely identified in meteorological, climatological and paleoclimatological research? This course will provide practical experience in the fundamental concepts, coding, use and abuse of two commonly used tools, empirical orthogonal function analysis and singular spectrum analysis. We will apply these tools to the analysis and interpretation of historical climate data sets. In parallel we will critically assess the strengths and weaknesses of these techniques by examining similar analyses published in the climate dynamics literature.

By the conclusion of this course, you should be able to demonstrate the following skills in pursuit of the following goals, via the following assignments:

Overarching Goals	Skills Goals	Assignments (see course structure)
Understand the strengths and weaknesses of tools which use empirically-derived basis functions.	Develop basic concepts in elementary linear/matrix algebra at the heart of empirical analysis techniques, and code scripts to perform these analyses in a high-level scripting language such as Matlab or GNU/Octave.	Prework, Classwork
Analyze and interpret the principal features resolvable in climate data (x,y,t)	Perform such analyses on historical gridded climate data (possibly to include sea surface temperature, sea level pressure, and surface air temperature).	Prework, Classwork
Critically assess similar analyses in the meteorology and/or climate dynamics literature.	Practice elements of the peer-review process.	Prework, Discussion

Acknowledgements

Tim Shanahan (Geosciences, UA) gave me the idea for the structure and content of this course.
This course was designed during a four-day workshop sponsored by the National Association of Geophysics Teachers (NAGT) and the National Science Foundation (NSF). A 'systems' style diagram of the course structure is here.
Data for analysis is served from the LDEO Data Catalog using Ingrid, courtesy of Benno Blumenthal at the IRI.
Ideas for appropriate reading materials have been borrowed from Gidon Eshel's University of Chicago GeoSci 236, Data Analysis in the Earth Sciences (II): Applications to spatio-temporal data.
We will use a network concurrent academic classroom license from Mathworks for Matlab R2006a. This allows us to run up to ten MATLAB sessions on the clue lab computers.
We'll use the M_Map plotting routines (including modified m_pcolor.m function) built and distributed by Rich Pawlowicz from the University of British Columbia.
John Chiang (UC-Berkeley, Geography) revised this course for his own use in Spring 2005, and sent back excellent ideas for the 2006 iteration.

Help

Matrix algebra, Linear algebra:

Useful texts:

G. Strang, Linear algebra and its applications. New York: Harcourt Brace, 1988.
R.W. Priesendorfer, Principal component analysis in oceanography and meteorology. New York: Elsevier, 1988.
D.S. Wilks, Statistical Methods in the atmospheric sciences. New York: Academic Press, 2006.

Randy Richardson's Inverse Methods course notes from GEOS 567 are here (thanks: J. Canon).
A manual for SVD and EOF analyses of climate data, from H. Bjornsson and S.A. Vargas, is here (thanks: J. Canon).

Matlab:

Mathworks' web pages are here; complete documentation is here.
A list of matlab functions useful for this course is here (upon completion of Prework 1).
A short primer, courtesy of the Navy (and now via Charles Tier, Univ. Ill. at Chicago), is here.
Another longer primer, courtesy of Paul Fackler at North Carolina State University, is here (thanks: J. Cook).
Getting started with MATLAB (v. 6): A quick introduction for scientists and engineers, by Rudra Pratap (now in v. 7).

Found a better source of help? Let me know and I'll post it. Thanks in advance.

top

Course structure

Each week's work will consist of the following cycle:

Prework: summarize/evaluate assigned reading, to be completed prior to the weekly meeting and handed in during class following discussion.
Discussion: main points and open questions from prior homework and present prework, during the weekly meeting.
Preview: occasional short weekly lecture on fundamental principle(s) for the week.
Classwork: analytical (paper & pencil) and computational (simple, generalized) problem sets, to be begun during the weekly meeting.
Homework: completion of classwork; analysis/interpretation of classwork results, prior to next week's meeting.

Assessment of prework, classwork, and student projects will be according to a grading rubric. Read through this also for tips on how to make the most of assignments. I may anonymously post exemplary assignment solutions to the course webpages to give you examples of the kind of work I expect.

top

Feedback

This is a new course (second offering) and a new approach for me; there are likely to be many "bugs". Your patience may be required often. I will be looking for feedback from you as we go as to how to best tailor this course to your needs. Consequently we may have informal, anonymous, occasional 1-minute teaching evaluations and mid-semester evaluations, in addition to the usual end-of-course evaluations. Your candid and constructive observations are appreciated.

top

Schedule/Syllabus (subject to change)
Note: Scanned readings are password protected to meet copyright fair use guidelines. If you need the userid and password, email the instructor.

When Topic Reading
Prework
Preview Classwork/Homework

Aug. 23rd
Introduction and Big Picture; the CLUE Lab

Course overview, logistics; Get set up to compute with Matlab on CLUE Lab PCs Homework 0 (due Aug. 30th)
Solutions are here.

Aug. 30th
The art of programming Pratap 2002, excerpts (5 Mb pdf) Prework 1 (due Aug 30th)
A class-compiled webpage of functions you might use for classwork assignments is here.
Programming in the MATLAB environment; Elements of programming logic and style (notes)
Homework 1 (due Sept. 6th)
My script is here. An example of student work is here.

Sept. 6th
Basic matrix algebra manipulations and implementation in Matlab Strang, Ch. 1, pp. 1-9, 19-27, 42-48 (8.4 Mb pdf)
Optional: Elsner and Tsonis (1996) Ch. 1-2 (2.7Mb pdf) and Navy's Matlab quick reference Prework 2 (due Sept. 6th)
A student response is here.
Matrix algebra is your friend Homework 2 (due Sept. 13th)
My script is here. An example of student work is here.

Sept. 13th
Input: gridded historical sea surface temperature dataset Bottomley et al., (1990) (11Mb pdf) Prework 3 (due Sept. 13th) A student response is here.
Know your data; covariance estimation
Homework 3 (due Sept. 20th)
My script is here. An example of student work is here.

Sept. 20th,
Sept. 27th
Linear algebra review; EOF analysis (space)
Peixoto and Oort (1992), Appendix B (348k pdf) and
Weare et al. (1976) (686k pdf).
Prework 4 (due Sept. 20th). A student response is here. A link to a Mathworks piece on applications of the SVD is here. EOF analysis of a covariance matrix (notes).
Homework 4 (due Oct. 4th)
Pencil and paper solutions are here; Matlab script is here; plots are here (Q3 plot; SST field evals, evecs, PCs). A student response is here.

Oct. 4th
Office hours this week are on Tuesday, 2-3pm, not Monday
Error, stability, truncation and filtering (space) Overland and Preisendorfer (1982) (313k pdf); optional: North et al. (1982) (348k pdf)
Prework 5 (due Oct 4th). A student response is here. Signal vs. noise: How many EOFs are interpretable? How stable are the interpretable ones?

Homework 5 (due Oct 11th)
Script to solve HW5 is here; my plots are here (Rule N results; filtering results; stability of eof patterns). A student response is here.

Oct. 11th
EOF rotation; Caveats on interpretation of EOF patterns Wilks (2006) pp.492-500 (3.3Mb pdf); Dommenget and Latif (2002) (641 kb pdf); Optional: A review and discussion of rotation methods by Richman and Jolliffe is here. Prework 6 (due Oct 11th). A student response is here. What are the pros and cons of EOF rotation?
Homework 6 (due Oct 18th) Script to solve HW6 is here; my plots are here (simple example; SST EOF pattern rotation (fig 1 and fig 2); corresponding PCs). A student response is here.

Oct. 18th
EOF analysis (time)

Mid semester course evaluation
results are here.
Ghil et al (2002) (621k pdf), sections 1-2.2 (pp. 3-1 to 3-11)
Prework 7 (due Oct 18th). A student response is here. Singular spectrum analysis algorithm; EOF analysis of a trajectory matrix; RC calculation. Notes from class are here (updated 10/23/06).
Homework 7 (due Oct 25th) Paper and pencil solutions are here; Matlab script to solve HW7 is here; Plot of results is here. Start thinking about the dataset you wish to analyze for your class project; a broad, searchable database is here. A student response is here.

Oct. 25th
Error, stability, truncation and filtering (time) Ghil et al (2002) (621k pdf), sections 2.3 (pp. 3-11 to 3-13); Vautard and Ghil (1991) (522 kb pdf); Elsner and Tsonis (1991) (297kb pdf)
Prework 8 (due Oct 25th). A student response is here. Monte-Carlo error estimates for eigenvector retention; effect of choice of embedding dimension
Homework 8 (due Nov. 1st) Matlab script to solve HW8 is here; plots of results are here and here. A student response is here.

Nov. 1st
Student projects: Introduction

student project consultations

Homework 9 (due Nov. 29th). A list of project topics is here.

Nov. 8th
Student projects: supervised worktime and consultation; Analysis of the covariance between two (or more) spatiotemporal fields PW9 reading: Wallace et al. (1992) sections 1-4 (1.5Mb); see Prework assignment for optional readings.
Prework 9 (due Nov. 15th). A student response is here.
student project consultations

Nov. 15th
Student projects: supervised worktime and consultation

student project consultations

Nov. 22nd
Thanksgiving week off
(office hours Monday, Nov. 20th as usual)

Nov. 29th Student projects: Presentations
Class meets at Tree-Ring Lab West Seminar Room 20, 1pm (map) which is in the basement of Mathematics East
Powerpoint presentations are here.

Independent project presentations and peer evaluations.

Dec. 6th
Summary and wrap-up
Mike is away Dec 4/5th (no office hours Monday)
Class meets at Tree-Ring Lab West Seminar Room 20, 1pm

Discussion: General results, lessons, caveats, future work.
Course evaluations.

When	Topic	Reading	Prework	Preview	Classwork/Homework
Aug. 23rd	Introduction and Big Picture; the CLUE Lab			Course overview, logistics; Get set up to compute with Matlab on CLUE Lab PCs	Homework 0 (due Aug. 30th) Solutions are here.
Aug. 30th	The art of programming	Pratap 2002, excerpts (5 Mb pdf)	Prework 1 (due Aug 30th) A class-compiled webpage of functions you might use for classwork assignments is here.	Programming in the MATLAB environment; Elements of programming logic and style (notes)	Homework 1 (due Sept. 6th) My script is here. An example of student work is here.
Sept. 6th	Basic matrix algebra manipulations and implementation in Matlab	Strang, Ch. 1, pp. 1-9, 19-27, 42-48 (8.4 Mb pdf) Optional: Elsner and Tsonis (1996) Ch. 1-2 (2.7Mb pdf) and Navy's Matlab quick reference	Prework 2 (due Sept. 6th) A student response is here.	Matrix algebra is your friend	Homework 2 (due Sept. 13th) My script is here. An example of student work is here.
Sept. 13th	Input: gridded historical sea surface temperature dataset	Bottomley et al., (1990) (11Mb pdf)	Prework 3 (due Sept. 13th) A student response is here.	Know your data; covariance estimation	Homework 3 (due Sept. 20th) My script is here. An example of student work is here.
Sept. 20th, Sept. 27th	Linear algebra review; EOF analysis (space)	Peixoto and Oort (1992), Appendix B (348k pdf) and Weare et al. (1976) (686k pdf).	Prework 4 (due Sept. 20th). A student response is here. A link to a Mathworks piece on applications of the SVD is here.	EOF analysis of a covariance matrix (notes).	Homework 4 (due Oct. 4th) Pencil and paper solutions are here; Matlab script is here; plots are here (Q3 plot; SST field evals, evecs, PCs). A student response is here.
Oct. 4th Office hours this week are on Tuesday, 2-3pm, not Monday	Error, stability, truncation and filtering (space)	Overland and Preisendorfer (1982) (313k pdf); optional: North et al. (1982) (348k pdf)	Prework 5 (due Oct 4th). A student response is here.	Signal vs. noise: How many EOFs are interpretable? How stable are the interpretable ones?	Homework 5 (due Oct 11th) Script to solve HW5 is here; my plots are here (Rule N results; filtering results; stability of eof patterns). A student response is here.
Oct. 11th	EOF rotation; Caveats on interpretation of EOF patterns	Wilks (2006) pp.492-500 (3.3Mb pdf); Dommenget and Latif (2002) (641 kb pdf); Optional: A review and discussion of rotation methods by Richman and Jolliffe is here.	Prework 6 (due Oct 11th). A student response is here.	What are the pros and cons of EOF rotation?	Homework 6 (due Oct 18th) Script to solve HW6 is here; my plots are here (simple example; SST EOF pattern rotation (fig 1 and fig 2); corresponding PCs). A student response is here.
Oct. 18th	EOF analysis (time) Mid semester course evaluation results are here.	Ghil et al (2002) (621k pdf), sections 1-2.2 (pp. 3-1 to 3-11)	Prework 7 (due Oct 18th). A student response is here.	Singular spectrum analysis algorithm; EOF analysis of a trajectory matrix; RC calculation. Notes from class are here (updated 10/23/06).	Homework 7 (due Oct 25th) Paper and pencil solutions are here; Matlab script to solve HW7 is here; Plot of results is here. Start thinking about the dataset you wish to analyze for your class project; a broad, searchable database is here. A student response is here.
Oct. 25th	Error, stability, truncation and filtering (time)	Ghil et al (2002) (621k pdf), sections 2.3 (pp. 3-11 to 3-13); Vautard and Ghil (1991) (522 kb pdf); Elsner and Tsonis (1991) (297kb pdf)	Prework 8 (due Oct 25th). A student response is here.	Monte-Carlo error estimates for eigenvector retention; effect of choice of embedding dimension	Homework 8 (due Nov. 1st) Matlab script to solve HW8 is here; plots of results are here and here. A student response is here.
Nov. 1st	Student projects: Introduction			student project consultations	Homework 9 (due Nov. 29th). A list of project topics is here.
Nov. 8th	Student projects: supervised worktime and consultation; Analysis of the covariance between two (or more) spatiotemporal fields	PW9 reading: Wallace et al. (1992) sections 1-4 (1.5Mb); see Prework assignment for optional readings.	Prework 9 (due Nov. 15th). A student response is here.	student project consultations
Nov. 15th	Student projects: supervised worktime and consultation			student project consultations
Nov. 22nd	Thanksgiving week off (office hours Monday, Nov. 20th as usual)
Nov. 29th	Student projects: Presentations Class meets at Tree-Ring Lab West Seminar Room 20, 1pm (map) which is in the basement of Mathematics East	Powerpoint presentations are here.		Independent project presentations and peer evaluations.
Dec. 6th	Summary and wrap-up Mike is away Dec 4/5th (no office hours Monday) Class meets at Tree-Ring Lab West Seminar Room 20, 1pm			Discussion: General results, lessons, caveats, future work. Course evaluations.

top