Representing Dynamic Arrays in VB.NET

Bill H wrote:
> Gene:
>
> I've been using mv.NET for awhile on UniData and have found the
> communication speed refreshingly fast. I use UO.NET through mv.NET and the
> speed was a huge increase over our use of PDP.NET and our testing of ODBC on
> D3 and OleDB on U2.
>
> But then we're just sorry-a$$ed users, not the oracle's of CDP. :-)
>
Well you know, I'm willing to believe that it's my specific configuration
requirements and not the product as a whole. If everyone saw the performance
that I did, nobody would use it.

g.

Tony Gravagno wrote:
>
> Gene posted a couple notes here about issues with the D3CL and tried
> mv.NET at my recommendation. We had a few rounds at it, and for some
> reason performance on his system really was unacceptable so he stuck
> with the D3CL. I can't blame him at all.
>
I do want to commend Tony for the effort he spent trying to figure out what
on earth was going on with the system. It was like there was atomic sludge
in the pipe or something.

g.

Bill H wrote:
> (MV & .NET) place you in a unique position to integrate both technologies,
> but I'm sorry to say this capability is nowhere to be found in the general
> .NET developer community.
>
Now just a minute. You might want to modify that to "found in the
general .NET developer community that learned software development with
..NET"

I suspect that most (if not all) of us here started our careers when the
world was young. I will agree strongly that the new breed of programmers
out there that start out in .Net are as useless as brake lights on a barbecue.

A lot of the problem is that they use the tool without understanding what is
going on under the hood and that's a huge problem.

I rely heavily on back-end processing for all my applications - it's a lot
more efficient to make a rule module do the heavy lifting and cough back
the answer than it is to do it on the client. In order to improve the speed
of my cellular modem based users, I even started inspecting transactions
with WireShark to see exactly what I could cut and still get the job done.
That's when I found out things like the D3 object library making calls to the
host for every IConv() and OConv() calls. I created "local" versions of those
just to be able to cut down on the packet traffic. The down side is that I can
still tell exactly what the D3 object library is doing by seeing the ASCII
dumps of the TCP/IP packets.

g.

Ross

I was simply highlighting the normal worst-case scenario for having
the business logic on the client end of the system.

No question that the user-interface should allow verification of data
entry as you go along, but even this can be improved by this
process.

Take the example of verification of a customer code. It might be
that to verify the customer code simply needs a yes/no verifcation and
ther customer name displayed. Even if this is a single read, for
many implementations (I don't know about mv.net) of client/server
communication this would need 2 calls - one to open the file, one to
read it. Then the whole record needs to be returned to the client in
order to extract the name. A simple subroutine call could return a
two lined variable - one indicating whether the customer code is valid
(a 1/0 true/false setting) and the second containing the name for
display.

Regards
Simon

On Aug 18, 12:48*am, Ross Ferris wrote:
> On Aug 17, 7:14*pm, Simon wrote:
>
>
> : Which involves 6 interactions with the server which are all "low-
> : level" read-writes. *It is better to simply write a single
> transaction
> : which simply calls a databasic program on the server to "Update
> : Customer x With this sale" which does all the 6 steps in one go and
> : returns the success or error. * *
>
> I agree to a point, BUT the reality is that I would expect that the
> ferification of a customer code would take place erly on in the
> invoice entry process --> I want to not only KNOW that I have entered
> a valid customer code, but also visually VERIFY I have selected the
> RIGHT customer by seeing name/address details etc displayed. Likewise,
> If I'm building up a stock invoice, I'm going to want to see that I've
> selected the right product line at a time, and also probably access m
> y host based pricing rules which can be "complex", including quantity
> breaks, mixed product discount dealks etc
>
>
> : * *I'm also going to represent the dynamic array as a
> : *string and use dotnet's extremely fast string handling code to
> : *manipulate it.
>
> When the compiler people get around to taking advantage of the new
> instructions available on the next generation of Intel chips to be
> released (which includes support of string searches as a single
> instruction at the silicon level --> vultures anyone?) this will only
> get faster

Sholom wrote:

>Does anyone have any good ideas about the best way to represent a
>dynamic array in VB.NET?
>
>I'm looking to use it as a temporary, in memory store, not as a complex
>database.

I just wrote something up. The thing to understand with this first
attempt is that there are two data types, Strings for Data and
MDArrays for dynamic arrays. In MV we look at these as inter-related
but I have separated them out. In MV we can say:
item = ""
item<2> = abc:@vm:def
We see here that a dynamic array is created with no data. Another
dynamic array element <2> is inserted inside the first, this consists
of two elements. The third step, is that the elements of dynamic
array in <2> are assigned with values - <2,1>="abc" and <2,2>="def".

Let's see how that's been implemented. This is actually functional C#
code, no shortcuts, magic, or imagination involved.


MDArray a = new MDArray();

This is the same as saying item="" but I'm defining 'a'
specifically as a dynamic array and not a String.

a.Insert(1 , new MDArray()); // a<1> is empty

We're inserting a new array inside the first element of the
existing array. This is recursive. "a" is a dynamic array and here
we're creating an element a[1] that itself is another array - but
dynamic arrays here can contain both arrays and data as we'll see.

a[1].Insert(1 , "first"); // a<1,1> = "first"

This is where string data must be separated from structure. The
array element a[1] contains a dynamic array but now we're also adding
an array of data. The first element is the string "first".

a[1].Insert(2 , "second"); // a<1,2> = "second"

There are dozens of functions in addition to Insert to manage where
data is placed, searching, deleting, and manipulation of data
elements.

a[2].Insert(1 , "prima"); // a<2,1> = "prima" (auto create a<2>)

Note that I didn't need to Insert a[2], just like MV, simply
referencing it expands the dynamic array to accommodate the value.

a[1].Insert(2 , new MDArray());

a<1,2> is a new array of subvalues, the existing a<1,2> shifts down
just like in MV. You should recognize here that unlike MV the a[1]
array element contains both a sub-array with more dimensions, and also
an array of data.

a[1][2].Data[2] = "sub"; // a<1,2,2> = "sub"

Note that the data elements dynamically expand to accommodate the
index specified, just like in MV. The Insert function isn't really
necessary.

Retrieving data:

string result = a[1][2].Data[2];

Some distinction has to be made between data and structure. When we
want data we need to be explicit. a[1] returns an array. a[1][2]
returns the second array element within a[1]. To get the data out of
that array, we need the .Data property, and then we need to specify
which element from that _data_ array we want.

Taking it to extremes:

a[1][2][2][4][1][3].Data[2] = "wow";

No inserts required, that one line creates a 6 dimensional array.
We can have as many elements as we want, and every single element and
sub-element has it's own array of data. (Starting to look like Caché
here...)

string final = a[1][2][2][4][1][3].Data[2];

That simply returns the data from the element just created. The
final value is "wow".


I need to hone it up a bit but now I'm wondering what to do with it.
Thoughts that come to mind are :
- See if there is a way to eliminate .Data for a[1][2] = "val".
- Support data types other than string.
- Serialize/save the data to disk for later reference.
- Try various functions and make sure they work.
- Test with VB.NET.
- Document, package, and sell it.

T

Tony Gravagno
Nebula Research and Development
TG@ remove.pleaseNebula-RnD.com

Simon wrote:

> Actually, I was about to start re-writing my vb.net based dynamic
> array code and was also thinking about how to represent the data
> structure. I'm also going to represent the dynamic array as a
> string and use dotnet's extremely fast string handling code to
> manipulate it.

Just wondering. When you say "extremely fast string handling" are you
refering to the standard String class, or to the StringBuilder class?

Hmm. Theory is proved once again: Nothing kills a thread more
brutally than a complete solution.

T


>Sholom wrote:
>>Does anyone have any good ideas about the best way to represent a
>>dynamic array in VB.NET?
>>
>>I'm looking to use it as a temporary, in memory store, not as a complex
>>database.

>I just wrote something up.
[snip]

On Aug 25, 4:15*pm, sh wrote:
> Simon wrote:
> > Actually, I was about to start re-writing my vb.net based dynamic
> > array code and was also thinking about how to represent the data
> > structure. * *I'm also going to represent the dynamic array as a
> > string and use dotnet's extremely fast string handling code to
> > manipulate it.
>
> Just wondering. When you say "extremely fast string handling" are you
> refering to the standard String class, or to the StringBuilder class?

stringbuilder class..

Simon

On Aug 26, 8:52*am, Simon wrote:
> On Aug 25, 4:15*pm, sh wrote:
>
> > Simon wrote:
> > > Actually, I was about to start re-writing my vb.net based dynamic
> > > array code and was also thinking about how to represent the data
> > > structure. * *I'm also going to represent the dynamic array as a
> > > string and use dotnet's extremely fast string handling code to
> > > manipulate it.
>
> > Just wondering. When you say "extremely fast string handling" are you
> > refering to the standard String class, or to the StringBuilder class?
>
> stringbuilder class..
>
> Simon

hmmm I seem to be answering thread items twice! Apologies.

Tony Gravagno wrote:
> Hmm. Theory is proved once again: Nothing kills a thread more
> brutally than a complete solution.
>
> T
>
>

I'm not sure the thread died because of a complete solution. It might
have died because people didn't understand what you wrote. (I, for one.)

1) Is this the complete code? I was expecting to see some code for
methods to implement the solution. Is the complete code on your blog?
Are you implementing dynamic arrays without any new code, just using
regular c# methods?

2) You create "MDArray a = new MDArray()". What is an MDArray
(multi-dimensional array?)? Is it a built-in type or is it something
you've implemented? I did a search on the term, and could only come up
with the fact that an MDArray is a SafeArray (whatever that is). Is an
MDArray what's called a jagged array in VB?

In short, your reply was over my head. Maybe you can expand on it in
your blog and post a link to the explanation.

It also might be a good idea to do the code in VB. Most of us are Basic
programmers, and while I have some grasp of c# code, in many cases (and
this I believe is one of them) the devil is in the details.