Overview

As a community, TERENA and its members produce a wide range of outputs from documents, advisories, software products, tools, right through to services.  Whilst many of these find a natural home and sustainability model through the development cycle of the output, the process is varied and can leave some ideas floundering. This is particularly true of software developments.  Most TERENA developments have a software component at the heart of the delivery - in NRENum.net it is the DNS Crawler, in TCS it is Djangora / Confusa - or have developed a service out of a software concept such as Filesender or Foodle.  Ensuring these essential tools are properly maintained should be a priority.

There are broadly four methods of funding for software outputs used by TERENA at the moment:

  • Contributions.  This is the REFEDS funding model where groups with a common interest club together to fund activities.  REFEDS allows for both ‘pool’ contributions and targeted contributions.
  • Service fees. In this model maintenance and small developments of software are funded as part of service fees charged directly to the community.  These can either be fixed or variable depending on the size of the NREN / userbase.
  • Project funding.  Several software outputs are supported by recurrent project funding (typically GEANT funds).
  • Social NREN. Some NRENS have made services they find useful openly available to the community at large to use.  The overheads for the software element are subsumed by the NREN in question.

An attached list shows how a variety of existing tools are supported. 

It would be good to establish a process where a clear path - from people in the community ‘having an idea’, as celebrated at TNC2012, through to implementation of an output or service - was established within the software development space.  Where as some of the steps within that process are well documented and implemented, other areas could be improved in terms of decision-making and clear routes for support – including funding models. 

The attached diagram shows a potential workflow from decision through to sustained output.  This diagram reflects one potential ideal and does not suggest that TERENA should move forward with all areas of work needed to achieve this vision.  This is an opportunity to reflect on the lifecycle and whether the one shown below matches TERENA community requirements.   Areas where work could be carried out as part of a pilot are highlighted on the diagram.

Most importantly, the diagram recognizes that there is no one size fits all model for outputs.  Some ideas might not go through the whole process described, and there are several sustainability decisions that could be made along the lifetime of the output.  

Within this workflow, software outputs are an area of current concern.  Whereas simple outputs such as white papers can quickly and easily be hosted by TERENA, support for development, maintenance and hosting of software is a naturally more complex process for which there are no clear processes in place.  This proposal aims to address that requirement.

 

Current Obstacles

There are a number of obstacles for software development in the current TERENA environment that could be addressed by this proposal.  The most immediate are:

  • Moving on from pilot / incubation.  The NREN community as a whole is very good at quickly turning ideas in to demonstration software and promoting the software out, but this often results in this pilot becoming a de facto service within the community without a clear support route and with an overhead on the original developer(s) unsupported by funding. 
  • Messy hosting arrangements.  There is no one place for software hosting, and as a result projects will often accept any offer of space or support they can get – leading to outputs being randomly distributed.  Although github is becoming more of a focus place for outputs, these are typically hosted on the repositories of individual developers to avoid costs.  This in turn creates a problem when developers move on or become refocused.
  • Developer pool is small.  It is becoming increasingly difficult to identify developers who can take on the maintenance and further development of outputs when initial developers move on.  Developer resources are now often outside the NREN community, creating the need for a more contractual approach to work.

 Aim

The overall aim of this proposal is to move the TERENA community further towards a model where its ideas and outputs relating to software development can be sustained in the most appropriate way.  It focuses on some short to medium term objectives to be undertaken within TERENA to support this goal.

The original problem statement was set as follows:

 

TERENA member NRENs and connected organisations have produced a number of software packages that have now gone into common use in the community. Many of these packages have been declared as “open source” with the appropriate licences. Considerable effort has been made in developing and deploying these packages and there is a non-trivial requirement for ongoing maintenance and support for the packages. In the IT world, nothing is static and software requires constant modification as systems develop around any one component. Many of the packages or systems are relatively small and would not merit substantial administrative overhead in themselves. However there are a number of things that need to be performed to keep the software up to date and viable. Bugs need to be fixed; additional native languages need to be added etc. The software is all NREN related software, but the user community is greater than the NREN connected institutions. It would include independent research institutes as well as the general public.

 

Objectives

The following objectives are proposed for a phase 1 project to address this aim.  This proposal is not for a solution service for software sustainability but a set of discrete tasks to define what TERENA should be offering in this space. 

  • Define support models: better define the different support types described as ‘listed’, ‘hosted’, ‘maintained’ and ‘active’ in the diagram at Annex 1 and create proposals for each support type.
  • Explore crowd-source funding / decision tools: Explore the opportunities for creating a ‘kickstarter’ type portal for community driven fund gathering, looking at open tools such as Selfstarter or services such as the JISC Elevator approach.
  • Improve current hosting: establish a central TERENA space in github or explore alternatives such as a hosted Jira for the TERENA community.
  • Explore software maintenance options: look at models to support maintenance more effectively (company contract, and propose a future solution for TERENA.
  • Review skills shortage: look at options for TERENA to be more involved in supporting seedcorn initiatives such as working with university students and better relationships with commercial suppliers.  

 Workplan [TBD these are just notes on important areas at the moment]

Need to think about what could be achieved in each area for example is objective 2 just a report on options, a test of platforms, a small scale pilot etc?

Work Item 1: Define Support Models

Expected deliverables: defined set of support models and TERENA service role under each one (report).  Recommendations for staff training / skills requirements to support these models.   Name and logo for service?

  • Listed – helping people to find tools, helping people to promote tools.  Is this a marketplace for TERENA? A broker?
  • Hosted – better support for repository tools, bug / issue tracking.
  • Maintained – supported by TERENA accredited developers? TERENA managing contracts on behalf. 
  • Active – this isn’t the right label….but meant to distinguish something that is in full on development mode.

 Would be important to include a review of legal advice for projects is given under this work item.  This could be something very simple, from sending TERENA staff on some training courses to support knowledge and advice, right up to use of lawyers as part of the support model.

This section should really look at what TERENA can bring to the table in this role.  As a:

  • coordinating function;
  • place to hold funds / assets;
  • place to disperse funds;
  • support for legal advice;
  • ability to hold contracts.
  • marketing support?? (can we do this?)

What about auditing roles?

Work Item 2: Explore Crowd-source funding / decision tool / fund management at TERENA

Expected deliverables: download and test functionality of available crowd source tools.  Report on findings and how this could be used in a TERENA workflow.

This requirement is about both fund raising and fund distribution. Process recognizes that this might not be what is wanted – a listed service may just be about making the right connections. 

It will be important that there are different ways of putting money in – so an organization could contribute something to the general pot, but could also specific that this must be used for project x, and the funds are ringfenced for this.

There would need to be a percentage cut to support TERENA doing this.  Would the community be happy with this?

Work Item 3: Improve current hosting offer

Expected deliverables: solution for hosting chosen and existing projects migrated where possible.  Templates for contractual arrangements. 

Peter has set up a TERENA space on github and and TERENA has approval to pay for licensed version of github as part of the TCS discussions.  There is also geantforge, assembla plus a whole bunch of stuff on NREN wikis / servers etc.

Need to look at what a good solution for this proposal.  Is buying a github licence worth it? Assembla? Atlassian tools??

Work Item 4: Explore software maintenance options

Expected deliverables: review of current contractual arrangements and recommendations for future models for software maintenance (report).

Need to look at current arrangements for things like Confusa and PEER – are these working?

Should we look at a pool of accredited developers (akin to accredited trainers for TRANSITS)? How would this work

Work Item 5: Review skills shortage

Expected deliverables: ??

Notes from Peter: among the obstacles, the problem is not just that the developer pool is small, but the developers are not affiliated with NRENs. In case of TCD the developer is commercial, in case of OER portal the developer is a contractor of GRNET (semi-commercial), in case of DNS crawler the developer is from a university, in case of NOC tools the developers are mostly from other communities (non-NRENs), in case of testbed provisioning tools the developers are from research institutes such as CESCA and others. To my experience, NRENs have no such software developers aboard who can really lift up a tool from pilot/prototype stage to production quality. In most of the cases NRENs have to find commercial partners and raise money to pay them! That is where TERENA should provide support.

Notes from Brook: look at running an event similar to the Google Summer of Code.

Timescales and Resources

 TBD.

Dependencies and Relationships

Due to the community focus of this proposal, there are natural links to discussions and work taking place elsewhere within TERENA.  Some of the obvious links are:

 

 

 

 

  • No labels