In definitions, always use the first rule that applies.

So, you probably know the 2-variable Veblen function, φ(α,β), or φα(β), but I won't write that, since it'll be used for something more powerful.

Definition:
φ(0,α) = ωα
φ(β,α) = (1+α)th point of γ↦φ(δ,γ) for all δ<β (β>0)

But you can also extend the function to 3 variables, like φ(α,β,γ). Like this: φ(1,0,0) = Γ0 = φ(φ(φ(...,0),0),0) = φ(0,φ(0,φ(0,...,0),0),0), φ(1,0,1) = Γ1 = φ(φ(φ(...φ(φ(1,0,0)+1,0)...,0),0),0), φ(1,1,0) = φ(1,0,φ(1,0,φ(1,0,...))), φ(2,0,0) = φ(1,φ(1,φ(1,...,0),0),0), φ(ω+1,0,0) = φ(ω,φ(ω,φ(ω,...,0),0),0), etc.

Definition:
φ(0,0,α) = ωα
φ(γ,β,α) = (1+α)th fixed point of δ↦φ(γ,η,δ) for all η<β (β>0)
φ(γ,0,α) = (1+α)th fixed point of δ↦φ(η,δ,0) for all η<γ (γ>0)

Of course, you can have φ(,...,α3210). So φ(1,0,0,0) = φ(φ(φ(...,0,0),0,0),0,0), φ(1,1,0,0) = φ(1,0,φ(1,0,φ(1,0,...,0),0),0), φ(2,0,0,0) = φ(1,φ(1,φ(1,...,0,0),0,0),0,0), φ(1,0,0,0,0) = φ(φ(φ(...,0,0,0),0,0,0),0,0,0), etc.

Definition:
Let Zk denote a list of k zeros (if k is negative, it removes the -k zeros after it.)
Let # denote an arbitrary list of ordinals
k is an integer greater than -1.
φ(α) = ωα
φ(Zk,#) = φ(#)
φ(#,β,Zk,α) = (1+α)th fixed point of γ↦φ(#,δ,γ,Zk-1,0) for all δ<β (β>0)

The limit of this, φ(1,...,0,0,0) is the Small Veblen ordinal, equal to ψ0Ωω) in Buchholz's function. To go further, we need a notation for writing numbers at transfinite positions. Here is the common one, known as the Schutte Klammersymbolen: ( αnβn ... α2β2 α1β1 α0β0 ) means α0 at position β0 (counting from 0 and starting from the left), α1 at position β1, etc, until αn at position βn, where n is finite. As an example, ( ε03 ω1 ) = (ε0,0,ω,0) - unspecified positions are filled with 0.

Definition:
"..." can be anything, even empty.
In a ( αnβn ... α2β2 α1β1 α0β0 ), any columns of the form { 0α } are removed (any references to them in the rules return 0.)
φ( α0 ) = ωα
φ( ... αβ γ0 ) = (1+γ)th fixed point of δ↦φ( ... ηβ δλ ) for all η<α, λ<β

The limit of this, φ( 1φ( 1φ( 1... ) ) ), is the Large Veblen Ordinal, ψ0ΩΩ) in Buchholz's function.

Dimensions

Now, the idea is that so far, all of our inputs to φ have been along a single dimension. So we can add a second dimension, so inputs could be along rows. So how would we write this in the Klammersymbolen? Well, make the last row a co-ordinate, so ( 1(1,0) ) is a
1 in the second row, in the first entry of that row, ( 1(1,1) ) is the second entry of the second row, ( 1(2,0) ) is the first entry in the third row, ( 1(ω,0) ) is the ωth row, even ( 1(1,0,0) ) - second plane,
( 1(1,0,0,0) ) - second realm, even nesting the notation to make ( 1( 1ω ) ) - ωth dimension, and even ( 1( 1(1,0) ) ) - second plane of dimensions. φ( 1( 1( 1... ) ) ) is the BHO, ψ02) in Buchholz's function. The formal definition is a bit too long to write here, and can be found in this blog post.

Beyond dimensions

Warning: Everything beyond here has not yet been formalized!

Version 1 - Multi-row Klammersymbolen

So, the array ( 1( 1( 1... ) ) ) is the least #↦( 1# ) fixed point - we could denote this ( 101 ). After this come things like ( 2( 101 ) ), ( 1( 101 100 ) ) - you can omit the zero in { 100 }, but it looks weird - then ( 1( 101 100 ) ),
( 1( 2( 101 ) ) ), ( 1( 1( 101 100 ) ) ), ..., and then ( 101 ), which is the second fixed point. Then there's ( 301 ), ( ω01 ), and ( 111 ) - the (1,0)-th fixed point, then ( 1( 1ω )1 ), ( 1( 101 )1 ). ( 102 ) is the fixed point of enumeration of fixed points of ( 1# ). Then ( ω02 ), ( 112 ), ( 1(1,0)2 ), ( 1( 101 )2 ), ( 1( 102 )2 ), ( 103 ), ( 10ω ), ( 10(1,0) ), ( 10( 101 ) ), and then fixed point of #↦( 10# ), which is ( 1001 ). Then ( 1011 ), ( 1002 ), ( 100ω ), ( 100(1,0) ), ( 10001 ), ( 100001 ), and nesting, (( 1ω 10 )). After that there's (( 1(1,0) 10 )), (( 101 100 )), ((( 1ω 10 ))), ((( 1(1,0) 10 )), etc. The limit of φ((((...)))) is ψ02Ω2Ω2).