Computer Science Senior Seminar
Project Abstract
Pick your religion:
Paper Outline
Pick your religion:
Project Overview
Pick your religion:
Paper (draft)
Pick your religion:
Project Todo
- Get Perl script working for database extraction/formatting.
- Determine which section of Cligen code I marked PARALLELIZE can
actually be parallelized.
Project Journal
- 2005-09-12
-
- I've been in contact with one of the maintainers of Cligen.
He's cleared up some points for me. Cligen is actually a weather
generator, not a prediction engine. That is, it doesn't what weather
will occur tomorrow, but can give some models for what
could happen. I don't think this will screw me up too badly, though.
- The plan right now is to find the meat of Cligen and
move it out into a C source file. I can then parallelize that with MPI
easily, without needing to know much Fortran.
- 2005-09-19
-
- I think I've determined the proper format for the Cligen
station (.PAR and .GDS) files. The first line has the station ID, the
country ID, the station name, the latitude and longitude, and elevation.
The next lines cover every day available, with year/month/day, max. temp.,
min. temp., and precipitation. I guess I don't even need solar radiation
for this.
- I'm in the process of writing a Perl script that will extract
the daily values from the database and plop them in a flat file ready to be
added to a Cligen station file.
- 2005-09-20
-
- The Perl script is going to be the critical thing to get done.
Right now I'm trying to figure out an easy way to iterate by day in the
Postgres timestamp. Ideally I'd do this completely within Postgres, but I
need to figure out the right function or date output for that.
- 2005-09-28
-
- Using the internal date_part() function in Postgres, I've gotten my
Perl script to extract timestamps as seconds from the epoch. I'll be
able to use this to get minima and maxima over one day (86400
sec).
- I then use Postgres's to_char() to format properly each part of the date I
need (YY, MM, DD), and also the data (XXXXX.X).
- My next problem is going to be figuring out how often the rain
meter is reset. If it's not regular, I'll have to be careful how I
extract that data, as a reset in the middle of a day could potentially
give erroneous data.
- I will also need to calculate the minima and maxima temperature for
each day, either with Postgres MIN() and MAX() functions, or with Perl.
- 2005-09-30
-
- My biggest problem right now is finding sources for my
paper. There's some books I have on ILL, but all the useful articles I
can find on-line are in journals the college doesn't have access to. I
think weather forecast/modeling is more esoteric than I thought.
- 2005-10-03
-
- I've done a quick once-over on cligen.f looking for things to
parallelize. I added PARALLELIZE comments at those points, and I'll
concentrate on those for possibly moving them out into a separate C file
for linking.
- 2005-10-05
-
- Rather than get a Perl script format a file for Cligen, I'm
going to get Cligen reading directly out of the database. I need to
track down all points when Cligen reads a station file, and then
rewrite those subroutines (in C?) to read from the database
instead.
- I'm waiting on Mary Tamm to write the code that will zero out
the rain meter on a regular basis. I also have to write some code that will
interprety the random times the rain meter got zeroed out before.
- 2005-10-07
-
- I think I really need to get a book on FORTRAN. There's too
many functions I don't understand.
- Charlie recommends going with embedded SQL in C. Here's a link.
- 2005-10-10
-
- I'm running into difficulties with embedded SQL. I
think I need to understand the relationship between ecpg and the C
preprocessor better.
- 2005-10-11
-
- 2005-10-12
-
- I'm going to use Cligen's sta_dat subroutine as my hook
to SQL. It's where all the data is snarfed up from the station file.
- 2005-10-15
-
- I think looking at lines 815 (sta_dat call), 2381 (sta_parms
call), and 2704 (actually reading) will be useful for SQL
integration.
- 2005-10-17
-
- Two-directional C-FORTRAN communication ("Hello
World"). Call C function from FORTRAN with varchar, then
pass back to FORTRAN and print. Test char,int,array.
- Copy and paste sample file to make it bigger and run
longer.
- Find FORTRAN book in Wildman.
- 2005-10-24
-
- I've found many FORTRAN references in Wildman.
- After tracing the problem to a possible interaction
within C itself rather than at the C-FORTRAN interface,
I wrote an equivalent C-C program. After some consulting
with K and R, everything now works by passing pointers around
within function parameter lists.
- Next step is to rewrite FORTRAN to pass by reference
within the function parameter list.
- 2005-10-28
-
- 2005-11-10
-
- I'm working on concatenating an option-6 input file and
rewriting the years to generate a big enough input for gprof to be
useful. The problem is that around line 2316 in cligen.f the input
file ends up with a leading newline. This needs debugging.
- I'm looking at how I want to do the UI. I'm debating between
the AJAX paradigm and just straight-up Perl CGI.
- 2005-11-14
-
- I'm running into serious problems with FORTRAN character
strings. Every string has a leading newline prepended to it,
so none of the file I/O works properly. I'm trying to get a
so-called TRIM function working that will strip the leading
blanks and turn them into trailing blanks.
- 2005-11-15
-
- Turns out that the problem is actually syntax errors
within the input file. I'm working with a guy from Purdue
in track down what's going on.
- 2005-11-15
-
- Found a good article from Argonne, written in plain English
about parallelizing weather algorithms: "Regional Weather Modelling
on Parallel Computers". It's in CVSWeb in my BibTeX file.
- 2005-11-19
-
- I'm in contact again with Charles Meyer from the Cligen
project. He's giving me a hand with getting the input files to run
properly.
- I'm going to try to get the Forest Service input files
working, because those apparently have a lot fewer
errors.
- 2005-11-21
-
- I finally have gprof data. Preliminarily, I'm looking
parallelizing chi-squared, because that is used in
a number of functions and uses upwards of 75% of the
compute time.
- In parallel with that, I'm going to expand my data
set to refine how much time is spent in randomization/
chi-squared.
- 2005-11-23
-
- I've got good gprof data from a bazaar node, from a run
that was about a minute long. Still looks like ranset to
me.
- Chi-squared calcs are done with chicnt, which is a 3d
array. I need to figure out how to pass multidimensional
arrays between FORTRAN and C.
- 2005-11-28
-
- I'm trying to figure out what is easier:
declaring/allocating arrays in C or in FORTRAN.
- 2005-11-30
-
- 2005-12-07
-
- I now think FORTRAN OpenMP
with
Sun's parallelizing compiler is the solution.
- I have the database aspect mostly wired up. I've input
an Option-6 file into a Postgres database, and then outputed
it again. It all works, except for dates after 1900. I think
there might be a leap year issue here, but I'm not sure.
- Next step is to code up the CGI. It should allow outputing
the input file, outputing Cligen's output, and changing the
input database.
- 2005-12-08
-
- Using a TEMP database that's a copy of the original
for generating custom Option-6 input files was a mistake.
Transactions+rollback is much easier.
You can follow my progress from the CVSWeb
interface.
Many thanks to Charles Meyer of Purdue for his timely help in running
Cligen.