Using Comma to Strip Away Data Structure in METSIM

Using Comma to Strip Away The Structure (1 of 2)

I started doing "Data Structure Manipulation - 1 / 2 /3 " comments, but thats not going to work. Just about everything we do, using APL or just building standard models involves data structure manipulation.

So maybe now my comment subjects will be more obscure. We have been using the comma to catenate (or join) data, and using it with axis to specify how the data is to be joined. We can use the comma to catenate scalars to make a vector, vectors to make bigger vectors, vectors with vectors (with axis) to create matrices, and matrices to matrices to make bigger matrices etc. In some of the cases (scalars catenated to become vectors, vectors catenated with axis to become matrices) the complexity (or the rank) of the data structure increased. A scalar has no dimensions, a vector has 1, a matrix has 2 etc..

But the humble looking comma has more "APL Kung Fu" than that. If we say " , M " or literally translated from our previous introduction to comma " catenate Matrix" we end up with a vector of the elements of the matrix M. The structure has been stripped away, and all we are left with is the list of data elements, arranged from left to right (arranged in the direction of increasing number of columns).

This will work with a variable of any level of complexity, no matter how deeply nested. It will smash through submarine bunkers, trash through filing cabinets, find those hidden scalars, and when it has smashed everything down to get to the individual data elements it will sit quietly and re-arrange all the pieces in the correct order, from left to right, and then wait to know what you want to do next.

The official name for this is "Ravel". And as you may have guessed there is a whole lot of other tricks as the more powerful "Ravel With Axis". But looking again at the simple (rather terse) statement " , M " and reading it with what we know about comma's catenate skills, we can fill in some of the blanks and see it is really saying " Catenate (or join) the matrix M to nothing in a way that increases the dimension of an unspecified axis".

And we know that when an axis is not specified, the default for catenate is "along the direction of increasing number of columns". So " , M " is saying "join M to nothing in a way that creates an increasing number of columns" Fair enough, that is what it does. So it looks like "Ravel" is just "Catenate" doing things that are implied by us not (having to) provide any more information.

But we need to talk a bit about the "catenate to nothing" concept - it is very powerful. By the way, this is my interpretation - or explaining in simple terms - what it is doing. Officially in APL there is another concept and use of the word "nothing", but I think this is OK for this unofficial and rambling discussion to be a bit informal. APL is supposed to be fun. A scalar has no structure. It is a single entity. And "nothing" is .... well nothing.

By catenating ( or appending as an additional element of ) something to nothing we must have created something that has structure - that is not a scalar. So the humble comma has another trick. If S is a scalar, then " , S " is a single element vector made from the scalar S. The shapeless structureless scalar S has been catenated to nothing and now has the vector properties of being a list of item(s) which increases in dimension in the direction of increasing number of columns.

The fact that it was created by appending something to nothing doesn't bother APL, and it shouldn't bother us. The real point of this article is not about what you can do with commas - it is to help understand that APL may sometimes appear to be terse, cryptic, unreadable but it is very sensible, logical, and easily readable if you learn the basic principles. The succinct nature of APL is not because there are lots of single character operators - there are not many at all and they are easy to learn. The succinct nature of APL is because it is so simple and logical that there really isn't much to say to express all of the mathematical expressions and data construction when you have the right tools.

Cheers, Jim