• Created by Unknown User (dyer), last modified on Aug 02, 2011

ASPIRE Study candidate topic: "AAI, Middleware, Data deluge and related issues"

Please use this page to make you contributions and comments on why this is topic is important for ASPIRE to explore in detail and include in the final ASPIRE report (summer 2012).

dyer@terena.org, ASPIRE Project Manager

Discussion

  • No labels

4 Comments

  1. Unknown User (dyer)

    From: Gudmund Høst,

    For the substudy on Data deluge and associated studies, I would suggest also to address the challenges involved in interconnecting the data generators and data "banks" (of unprecedented size) with compute resources (also of unprecedented capacity, possibly located to use green energy, maybe even dynamically switching between most economic compute resources) and making these accessible to every European researcher.

    Best regards,

  2. Unknown User (m.norris)


    Last time around, we had the digital divide, now it's the data deluge.  I'm aware that alliteration like this can, like other clichés, get wide currency and usage, often with little thought as to its actual meaning.  I've seen "data tsunami" used as well, again largely for effect.

    The amount of data on the Internet is growing, and will probably continue to do so in the next few years.  The rate of change may be linear or exponential.  However, "deluge" and "tsunami" would signify a sharp discontinuity, when the amount of data increases by a great amount in a very short time.  Is this really what we are expecting, or will the change happen over a longer time?  OK, so we are concerned at the growth in the amount of data and the implied need to transport it over large distances, but do we need to use scare tactics to get the message across?

    I remember when we first started installing GE circuits in HEAnet, some emotive researchers said they would soon use all this bandwidth.  Eight years later and only one or two clients have shown that they can use more than 1Gbit/sec of external capacity.

    All the best.

    Mike Norris

    1. Unknown User (dyer)

      THANKS for the comment Mike. I really hadn't thought of the terms "deluge" or Tsunami" as being discontinuities and I don't think that is what anyone is predicting.

      When I look at the Compendium data the "basket of NRENs" total traffic seems to grow about 40% per year (doubling time about every two years). If that were to continue without the rate declining over the next 10 or so years then we would reach Exabytes of transfers (per year) before 2020. Maybe it will be less, maybe it will be more (smile)

      John

  3. Unknown User (alyall@ebi.ac.uk)

    I agree that it is more of the same in physics, materials science etc but there is a bit disontinuity in biology.  It will be caused by personal genomics (amongst other things) - data generation rates of one million times what we have now by 2020.    There are emerging "disruptive technologies" that are going to cause the discontinuity.   See the wikipedia article for example.