Thinking Beyond Excel for METSIM

Thinking Beyond Excel

Thought is limited to our range of being able to visualise and use things. I can't really see a string or parallel sheets model of the universe, but I am quite OK with the expanding balloon surface as my model for now - and I don't see anything I can do with any of these right now anyway.

But when it comes to data structure models, once you take the first step past the flat world of Excel (limited to rows, columns, sheets, and the intersections of these) - you will wonder why you ever put up with all the limitations that this imposes and how the need for real data handling somehow got lost in the pursuit of seemingless endless and ever changing cell formatting options, uncontrolled (or incomprehensible) gesturing transformations, the given today and then taken away tomorrow functionality - and huge and ever changing proprietary file formats.

OK - I'm sorry - I feel better now.

The answer is, of course to use Excel when it does the job OK (which is alot of the time) - and to use APL when you have real work to do. Maybe with time you will find it easier to use APL for more things, especially if you think you are going to want something to work for more than one cycle of MS self destruction and re-creation.

Taking the Excel data structures as a familar starting point (or a database etc.) - imagine you are building a data model that starts with rows and columns - then all is good in Excel. Then you need to insert a row, add a second sheet - yep, all good. But what if you want to insert a 10 x 10 array in one cell, or split out one column into 50 columns, or have an array in cell of another array, which is in the cell of another array etc..  Maybe you had a number, deeply nested, but now it has to be replaced by a word ( ore type "2" is now "Brecciated Boiling Zone"), and you don't want to have to re-program everything for the variable type change.

In the real world this is how knowledge accumulates. As we gain detail, increase our model complexity, and establish better understanding of the relationships between things we want to stuff more data into the logical place for it - even if we didn't plan ahead knowing something more would be required - and we don't want the data structure to explode in size by having to add the same "spaces" for all the other things. We don't want to become relational database experts - we want the "system" to work for us, not the other way round.

APL has the concept that data has both structure and content. A variable may have structure and no content (is ready to receive content), or the structure can be changed to suit the changing requirements. Complex data structures can be "encapsulated" - or "Enclosed" to become "simple" (stackable in neat arrays like a shipping containers) and can be "opened" or "Disclosed" so you can rummage around in the contents of a container, any of the boxes in the container.

Obviously the APL data structure and language is not the only way to handle real world complex data requirements, and Metsim and APL can be used seemlessly with all Microsoft Data Models and other systems - but APL is the easiest, and most accessible to us as Metsim users (not full time programmers or IT Systems Administrators).

Looking at data in Metsim using the APL Keyboard, and asking for the shape of what you are looking at (using rho, or Alt + R ) you can work out for yourself most of the snippets of code and expressions that users are sharing here - and more importantly you can check it and change it if required as Metsim evolves.

But nothing replaces proper training of how to use what is already provided in Metsim, and being able to get and change things doesn't mean it is always a good idea - particularly if it is being used as a "work around" for poor modelling practices.

Cheers, Jim