i'm developing a Flash application that needs very fast access to a very large
amount of data to represent in a dynamic visualization.

currently, i am planning to have the data parsed into several XML documents to
then load into Flash, and parsed into Arrays via ActionScript. it appears,
however, that i will need several arrays (about 12) that will each need to have
a length of 1825100 values!!!

i ran an experiement to see how flash would even react to one such array of
this length:

var clockArray = new Array();

for (i=0; i<1825100; i++) {
clockArray[ i ] = i;
}

i get the error dialogue box "A Script in this movie is causing Flash to run
showly ... do you want to abort?" ... several times before it even has a chance
to compile ... !

is there a maximum length to an Array in AS?

anyway, perhaps there is a better way to go about this? again, i need to have
very fast access to this large amount of data for immediate visualization
purposes. could Flash remoting handle this better? Shared Objects? or some
combination?

all insights and feedback are appreciated.


thanks,

~ M

Paging the data is a proven way to handle large amounts of data, isn't it? You
surely don't need milion of values at once? A handfull of them on screen at a
time should do. If you perform searches then it's best for you to do that on
the server in some server side scripting language (SSSL) like CFM or even
better in the database (stored procedures or Oracle PL SQL).

Why don't you parse pages of your data directly from a SSSL into an Flash
dataset (DS)? Then your data is always fresh and light. DS is in a way an array
and is easy to handle.

For visualisation you also don't need all the data at once. You can geta
chunk, draw it, clear the DS and get a new chunk.


Hope it gives you some ideas. Hear ya!


Best regards,
Nenad K.

yes, well, good insights ...

i do think using the DataSet component is the way to go (and not just Arrays).

with this particular application, however, i also think that i will need all
data (millions of elements) to be immediately available at one time (without
any potential latency of server-side communication):

here is what i am doing: i am creating an astronomical / astrological
application where i want to visualize the positions of the planets from 3000 BC
thru 2100 AD at variable increments of anything from 1 day to 1 year, to even
perhaps 100 years. the user will have the ability to control the real-time
forward/backward motion of the planets through time, and adjust the increment
of motion/time on the spot. so it seems that the most effecient way to
accomplish this, and have the visualization move instantly with such dynamism,
will require that i have the entire lot of positional data available for each
planet for each day (the most granual of incremental data). if the user
chooses a broader increment of motion (say, 20 years per advance cycle), then i
can simply skip through the mass of data at the approriate increment.

i have considered how server-side calls, or Flash Remoting could possibly pull
this off. calling in data as it is needed ... yet i do not want to risk any
latency issues during the visualization. and the intention for this
application is to be a stand-alone desktop application (not requiring on-line
access).

as of now, via a parellel dialogue on this issue (see:
http://www.macromedia.com/cfusion/webforums/forum/messageview.cfm?catid=288&thre
adid=1059479), i am consider a fairly dynamic method of bringing in the total
amount of data involving many looping mechanisms, the DataSet, and the
SharedObject.

[quoted text, click to view]
dataset (DS)?

is there a more direct way to put data into a DataSet than by first loading it
via an XML document, and parsing it in? can i say, connect a DataSet directly
from a server-side script, or via Flash Remoting, or Flash Comm Server?

thanks. and any more insights will be greatly appeciated.

DataSet has more overhead than array in flash. You will probably never be able
to get fash to loop that many times through any significant amount of data.
You need to break up the data. Also hash tables seem to work well when looking
up values in an array... thus avoiding search loops.

so in saying the DataSet has more overhead than an Array, you mean it is a
disadvantage to use a DataSet? or you mean it has more data capacity?

and yes, we are working on a breaking up the data loader though various loop
dynamics. a dialogue on that here:


http://www.macromedia.com/cfusion/webforums/forum/messageview.cfm?catid=288&thre
adid=1059479

what is a hash table? where can i find more information on that?

thanks

Hi Miquael,

ok, I was thinking about your project. You are actually creating a simulation
- something that exists in the real world - the motion of planets. In this case
I don't see it as a good way to go, getting the positions from a database. The
movements of the celestial bodies follow some very specific laws, which can be
expressed using functions, which I am sure you know of. You can find those
functions in books on astrophysics. You might also want to contact your local
astronomical club, they should help you finding this kind of information.

The error message you were getting when running the clockArray script, that's
a security limitation of Flash Player. If your script does not finish after 15
secnonds, you get this message, which prevents users from deploying infinite
loops. You can approve the script to continue running, but after 15 seconds
you'll get the message again.

So, if I were you, I would go for a real simulator - using functions to
generate positions of the planets. There might even be some libraries already
written for this, try searching for them on the Internet.


Good luck with your project,

Simon.

thanks for the thoughtful insight, Simon. yes, i will look into the
possibility of and consider using more fundamental astronomical functions to
drive the simulations. if this could work, it would definately be worth while.

thanks!