Today, complex applications have a demand for resources, computing power and storage. Cloud computing with the elastic and pay-as-you-go models changed the landscape. Moreover, organizations have resources and infrastructure (i.e. networking devices, gateways, wireless devices) that would like to combine with other resources. However, users are still locked-in by the resource vendors and face the problem of not easily specifying and operating scenarios involving resources by many resource providers. Either you would like to deploy a complex application over such a rich infrastructure, or you would like to just test you application in such a complex environment, you need to federate resources. Still users need ways to define complex deployments without knowing the underlying details.

Welcome to FSToolkit

“Federation is the act of combining data or identities across multiple systems. Federation can be done by a cloud provider or by a cloud broker. A broker has no cloud resources of its own, but matches consumers and providers based on the SLA required by the consumer. The consumer has no knowledge that the broker does not control the resources.” - Cloud computing community

Federated infrastructures in experimentally driven research need models, architectures and tools to address the definition and execution/operation/control of the experiment. In our work «Federation Computing: A pragmatic approach for the Future Internet» , we presented a paradigm called Federation Computing where it deals with the aspects of defining and operating/controlling experiment scenarios or so called Federation Scenarios in the context of the Panlab project. A Federation Scenario is a well-defined specification of (heterogeneous) services or resources and their configurations, offered by a diverse pool of organizations in order to form richer infrastructures for experimentally driven research. A Federation Scenario is an equivalent of a requested SLA which is required by the end-user, the customer of the federation. These federation scenarios represent customer needs such as i) evaluation and testing specifications of new technologies, products, services, ii) execution of network and application layer experiments, or even iii) complete commercial applications that are executed by the federation’s infrastructure in a cost-effective way.

Testbed experimentation needs to be applied to wider areas, such as the core research of enterprises and other social infrastructure services. It will be necessary to consider new infrastructures for supporting approaches that exploit, extend or redesign current Internet architecture and protocols. During the last few years experimentally driven research is proposed as an emerging paradigm for the Future Internet on validating through testing-scenarios new architectures and systems at scale and under realistic environments. Until recently, testbeds used in testing activities have usually a certain scope of testing capabilities. Organizations own resources and infrastructure (i.e. networking devices, gateways, wireless devices) that would like to offer and combine with resources of other infrastructures in order to enable richer and broader experimentation scenarios. Experimentally driven research addresses the need to evolve the test beds into coherent experimentation facilities. This is possible by enabling large-scale federated infrastructures of exposed organizational resources and testbed facilities. Such future experimental facilities are leaded by global efforts like GENI and FIRE .

The need for broader experimentation scenarios, with testbeds that complement each other, leads testbed owners to implement federation mechanisms. Thus, efforts like broader adoption of Slice Federation Architecture [3] have emerged. However, such federation efforts require from testbed providers to adapt new mechanisms for their testbeds. This means that usually owners should maintain in parallel two systems: the existing one that control and manages their testbed (i.e. users, reservation, resources, web access, etc) and the system that exposes resources to the Federation (like SFA). The latter means that the testbed owner should follow federation policies for the testbed’s resources. What happens, though, with cases where such federation is not possible? When users or resources cannot be federated? When a testbed owner cannot adapt his existing systems with federation mechanisms? When some policies of the federation cannot be followed by the testbed? This is more evident nowadays with the emerge of cloud computing testbeds, free or commercial ones. Still, in such cases like the above we will have the need to create experimentation scenarios that need testbed facilities from every domain.

Usually, experimental facilities expose Application Programming Interfaces (APIs) to users in order to allow them to reserve, configure and utilize resources in their experiments. Moreover each testbed trusts its users. These trusted users can utilize a testbed’s API via client tools. Inter-testbed experimentation scenarios involve scenarios that are created by a user that he is trusted by each and every testbed that participate in an experiment. These scenarios can be created, even if there is no prior federation among the testbeds. The federation is created “on-the-fly” on behalf of the user that all facilities trust.

Inter-testbed federation

FSToolkit enables Inter-testbed experimentation scenarios, ad hoc, on demand federation, reconfiguration and in general inter-cloud federation.

The figure displays how an inter-testbed experimentation scenario is envisioned. To enable inter-testbed scenarios, first of all, it is assumed that APIs are provided by the testbeds to end users. Such APIs might be for example: the MyPLC API , the SFA API or the OMF API . More over can be an emerged cloud API like an OCCI or EC2 compliant API.

Inter-testbed scenarios are meaningful in cases where forming a federation between involved testbed is not there . There are many cases where testbeds cannot join a federation:

-The experimenter owns a small local private testbed (ie in his university) that it is not possible to put it on a larger federation scheme (like SFA). Still the private facility has an API (eg XMLRPC proprietary, etc) that can be accessed by trusted users of the facility

-There is a testbed that is not part of any federation. For example it might be a commercial one. Still it is accessible to its trusted users via provided API

-Policies of a testbed (eg regional) restrict the testbed of joining a particular federation

-A commercial cloud provider that doesn’t want to join a federation.

As shown in figure, the experimenter is the entity that all testbeds, which participate in an experiment, trust. The experimenter has accounts and can access the various APIs on all testbeds. For example:

-for his private testbed the experimenter has a registration there and uses X509certificates to be trusted and simple http calls

-for another testbed the experimenter is authorized via username and password and uses SOAP call

-for accessing a testbed that belongs to a larger federation (e.g. SFA) the user just uses federation access methods.

Having access to all these testbeds via different authentication methods and API schemes, it is made possible that a Federation is created for this experiment on behalf of a trusted user. In such cases today the user should use different tools for each testbed and configure resources for each testbed. Then by applying proper configurations to each testbed (ie public IPs on machines, installing and configuring applications) can create the experimentation scenario. Of course many times there would be interoperability issues, but this is not a topic of this short paper.

FSToolkit make it possible to automate the whole process and enable the formation of an inter-testbed federation scenario.

Enabling inter-testbed scenarios

To enable and automate the definition of inter-testbed scenarios different authentication means should be handled within the tool by providing a seamless way. Then with one scenario definition the tool should act as a broker and access each testbed for configuring resources on behalf of the user.

The Federation Scenario Toolkit

The above Figure displays a federation scenario in the Federation Scenario Toolkit (FSToolkit) written in the Federation Scenario Description Language (FSDL). FSDLis a textual Domain Specific language that can be used by experimenters to describe experiments by choosing services and resources. It is based on the Eclipse platform. The meta-model that is the basis of FSDLis called Broker meta-model (called in the past as Office meta-model) presented in TheBrokerModel (TheOfficeModel) while its syntax can be found at FSDL. FSToolkit enables the definition of resource request scenarios, so called Federation Scenarios, agnostic in term of providers. To help the end-user with the syntax and protect from syntax errors the FSToolkit environment has a specific FSDLeditor.

In FSDL the experimenter can create an experimentation scenario that contains requested services or resources with their configurations. In the simplest usage an FSDLdefinition starts with the keyword RequestedFederationScenario? followed by a name. A set of import broker statements that contain definitions of the brokers (the resource brokers, services and resources) may follow. Next, one can define either a resource agnostic scenario request or specific resources of providers. To illustrate the above we will discuss an example.

Next Figure displays a scenario where we import resources by accessing 3 testbed providers:

• We import resources from the Panlab [10] project ,
• There are offered node resources from an SFA test site called PLA which belongs to a larger SFA federation.
• There is a private local testbed in the university called UoPPLC again running another version of SFA

A federation scenario on many sites

In this scenario of the above figure , we request resources from providers that each one has an API. The scenario requests 10 nodes from PLA, a node from UoPPLC and an echo service from Panlab. The user then can configure all these resources. Even with settings that one resource produces and another consumes. What is needed is to handle also the different identities of the user.

Manage multiple identities

Next Figure shows how multiple identities are managed within FSToolkit by using authorization pages. The tool can handle multiple accounts even if there are many facilities offering the same API. For example the user can configure multiple SFA accounts on many SFA enabled sites, public or private. Identity is used by the tool for accessing the facility on behalf of the user. Queries are made against the facility in order to find out offered resources to the user. Identity information is also used for provisioning the resources to each individual facility. If the facility offers enough detailed information regarding for example, resource constraints, reservation information or other policies, these are mapped to the editor in order to ease the description of the experimentation scenario.

Handling authorization on many testbeds

Provisioning

What is critical with the specification part of a federation scenario is the proper and valid configuration of the participating resources. While the scenario is operated by a customer (i.e. during an application deployment or during an experiment on the federated infrastructure), the federation must ensure that all SLA terms are fulfilled and nothing is violated or falls out of the scope of the SLA. To this end, the SLA must be constantly monitored for different aspects (i.e. metering, service quality, security, etc). This API is called Federation Computing Interface (FCI) . FCI is currently a Java implementation of a layer that allows with one API, access on many facilities via provided APIs. FCI wraps the different aspects of all APIs and bring them under a common model. Moreover, bringing programmability to describe experimentation scenarios, allows users to express really complex workflows. Even if there is a need for resource control during an experiment, FCI eases this process.

Enhanced tooling via plugins - Plugins enable access to testbeds

FSToolkit is based on the Eclipse platform and is being deployed to end-users as a set of plugins. FSToolkit is possible though to be extended through defined Extension Points. Figure 5 shows this concept. There are three main Extension Points in FSToolkit. The IWoklflowEngine and the IProvisioningJobListener are used by plugins that are capable of handling provisioning of resources. The third extension point, the IBrokerRepository, can be used by resource providers and brokers to expose resources to the end-users, in order to create federation scenarios. A provider in order to support provisioning of resources, the extension point IProvisionResource of the Provisioning Engines must be implemented. For example, in the SFA’s case a Plugin is built that is capable of transforming concepts of SFA’s (ie nodes, network, slices) to an equivalent expression of SFA’s in Broker model. Moreover, the provisioning plugin via the IProvisionResource extension point propagates commands to the FCI layer. The FCI has then an implementation of wrapping commands to SFA API.

Federations of Resource Providers and Brokers

A Federation Scenario describes end-user needs for services provided by resources of a federated infrastructure. At this point resource brokers play a key-role in creating and supporting federated infrastructures. A resource broker matches customer requested services and provider resources from the federation.

The FSToolkit is a development environment for specifying and operating Federation Scenarions. The editor is based on the textual modeling framework (TMF) of Eclipse and specifically the Xtext framework , which helps implementing rich editors by a definition of a specific syntax. The toolkit supports a family of Domain Specific Languages (DSLs), used by resource brokers, resource providers and experimenters, that have a defined meta-model, the so called TheBrokerModel, as an abstract syntax. The meta-model is also capable to support a federation of federations. This means that the model and therefore the DSLs are capable to describe federation scenarios involving resources even from a pool of resource brokers. The presented support language tooling is available for the Eclipse workbench.

An overview of the FSToolkit

---

An Overview

There are 4 steps to follow when developing a federation scenario with FSToolkit

1. Specify your scenario by requesting services

Use the FSDL syntax to describe your request of services

2. Get a proposed plan by the resource brokers about resources implementing your request

The ResourceAdvisor will suggest you plans that implement your initial request with available resources by resource provider

3. Submit your final scenario involving resources for provisioning

Your scenario, which contains now actual resources (computing, storage, etc.) is submitted to workflow engines for provisioning

4. Manage your resources

Within the toolkit there is an OperationView, where you can manage your resources

---

Download binaries and installation

DownloadAndInstallation - check this page for installing FSToolkit

---

Guides

QuickStartGuide - for a quick start

TheBrokerModel - read more about our meta-model

ProvidingResources? - read how you can expose your resources and provisioning mechanisms to FSToolkit users

ExtensionPoints - read how you can extend the toolkit with plugins

Testbed Management - an introduction and guide to PTM the Testbed Management and the Resource Adapters

FSDL - an introduction and guide to Federation Scenario Description Language

BrokerDSL? - an introduction and guide to Broker Description Language (old OfficeDL)

SDL? - an introduction and guide to Service Description Language

RDL? - an introduction and guide to Resource Description Language

ResourceAdvisor - a service matching services and resources

Federation Computing Interface (FCI) - An API for accessing resources of the federation

FCI Java example - An example using the FCI within Java

scheduleLab - Check our calendar of running experiments and reserved resources

FedWay - A gateway for federation API access.

SFA Client - Guide for writing a Java client for Slice Federation Architecture (SFA).

---

Examples

ToolkitScreenshots? - some screenshots

wiki:Ap2pScenario? - an example Deploying A P2P Scenario

---

Source code

Source code is available at github:

​https://github.com/ctranoris/fstoolkit

---

Contact

for any questions send an e-mail to tranoris [at] ece.upatras.gr

---

Some relevant conference papers

Christos Tranoris

«Adopting the DSM paradigm: defining federation scenarios through resource brokers for experimentally driven research», 1st IFIP/IEEE Workshop on Managing Federations and Cooperative Management (ManFed?), IFIP/IEEE International Symposium on Integrated Network Management, May 23, 2011, Dublin, Ireland (best paper award) ( ​http://dx.doi.org/10.1109/INM.2011.5990574 )

Christos Tranoris, Spyros Denazis

«Federation Computing: A pragmatic approach for the Future Internet», 6th ΙΕΕΕInternational Conference on Network and Service Management (CNSM 2010), October 25-29, 2010, Niagara Falls, Canada ( ​http://dx.doi.org/10.1109/CNSM.2010.5691310 )

---

trac Starting Points

• TracGuide -- Built-in Documentation
• ​The Trac project -- Trac Open Source Project
• ​Trac FAQ -- Frequently Asked Questions
• TracSupport -- Trac Support

For a complete list of local wiki pages, see TitleIndex.