Skip to content

GitLab

Commit 2dbe3fa0 authored by Kyriakos Kritikos's avatar Kyriakos Kritikos
Browse files
Options

Modifications to importer code

parent 65f42bdb
Hide whitespace changes
Inline Side-by-side
Showing with 265 additions and 36 deletions
README 0 → 100644
View file @ 2dbe3fa0
This is a maven project incorporating java code which can be used to import various types of models into a CDO Repository
by also respecting the model storage guidelines that have been specified in the context of WP4. This means that the code
is responsible for generating the respective CDO resource folder structure as well as importing the models into resources
which are stored at particular points inside this structure. Some of the CAMEL models might also be split into two or more
parts, according to the model storage guidelines, and stored in separate resources. It must be highlighted that if the user
modifies the inCDO boolean parameter given as input to the constructor of the main java class (ModelImporter) with the value
of false, then some of the methods provided by this class are meaningful to be called as they can be used to store some of
the basic models that are first generated into the file system. Thus, to conclude, the main class exposed can be used to generate basic
models and store them in CDO Repository or the file system as well as import provider models and use case models into the CDO
Repository.
FUNCTIONALITY: The following functionality is exposed:
(i) import provider models which have been placed at a particular directory in the file system into the CDO Repository
(ii) import use case models: currently only Scalarm is exploited as this is available in the jar file produced out of the
CAMEL domain code. If the git repository containing all use case models is exported as a jar file, then this method will be
modified to load any use case model from this jar file and import it accordingly to the CDO Repository according to the
model storage guidelines.
(iii) import geo location models: here a specific geolocation ontology is loaded and processed in order to populate a
location model with geolocations and then this location model can be stored in CDO Repository or exported into the
file system
(iv) import (basic) security model: here there is a specific code which generates the basic security model based on
particular input already available and then stores it in CDO Repository or exports it in the file system
REQUIREMENTS:
- Java 7
- maven
BUILDING:
run "mvn clean install" to compile the code and produce the respective jar file
CONFIGURATION:
There is no configuration for this code but as it depends on the CDOClient code, the respective .properties file
of the latter code (eu.paasage.wp4.client.properties) should be either pointed out by the PAASAGE_CONFIG_DIR system variable or the
pointer to its encompassing directory in the file system can be specified in the command line in the end by
providing the following String: "-Deu.paasage.configdir=<directory path where the properties file resides>".
The respective README file of the CDOClient in the respective git repository can also be inspected.
USAGE:
This component can be exploited/used through the following ways:
(1) via command line by running either:
(a) "mvn exec:java" or
(b) going to the "target" directory and running "java -jar importer-<code_version>-jar-with-dependencies.jar"
where code_version is the current version of this code.
(2) importing the contents of the respective directory in the Application Development Environment of your choice and then running "mvn exec:java" to run the code's main method. This exploitation route should probably concern just testing that the CDOClient code runs smoothly. Towards this goal, the Sens_App_Scenario.xmi file has been created in the "examples" directory which can be used to load a particular CAMEL model and then store it in the CDOServer. This file is used by the main method of the code to show that the loading of XMI files is possible and then the CDOClient can be used to store the loaded models. If you desire to load another file, then you will have to either place it in the "examples" directory or another directory of your choice but of course also change the main method to point to the appropriate file path. The main method also produces as output one XMI file which maps to a model/object produced via a particular query;
(3) produce the jar of the code via "maven clean install" and incorporate it in the libraries of your code (obviously change pom.xml file to include the dependency if your code relies on maven). In this second way, probably you need to follow a particular process through which you can exploit the code as there is no sense any more in exploiting the main method. Actually, the main method can be used as a guide based on which you will be able to exploit the CDOClient. A more involved guide and detailed documentation is provided at the Documentation.doc file.(you can also run the code's main method at the command line by issuing.
There are two options for using the Importer:
(a) for showcasing reasons to see the exhibited functionality
(b) via a command line-based manner
If the first option is selected, you just execute the code (main method of ModelImporter.class) and see
the results in the CDORepository.
If the second option is selected, then you can individually run particular methods of the ModelImporter
in order to import different types of models either in the CDORepository or the file system.
To proceed with this option, you need to provide 1 or more arguments to the main method of ModelImporter.
This can be done for the first way by appending in the end of the command line the following
String pattern: "-Dexec.args="<method_name> <arg1> <arg2> ... <argN>" where <method_name> is
the name of the method to call and <argi> is an input parameter that can be provided to this
method.
For the second way, you provide the respective main method arguments through the corresponding
way provided by the Application Development Environment of your choice.
In either case, some particular details need to be known in order to have the desired results.
First of all, as many models depend on the basic location model that we can import, it is recommended that
this basic location model is inserted in the first place. Otherwise, the importing into the CDO
Repository of other type of models will fail.
Secondly, each method comes with different number of parameters while some methods also need to
set also the boolean attribute of the constructor of the ModelImporter (in order to know whether
we need to import the respective models in the CDORepository or in the file system).
To this end, depending on the amount of main method arguments, the method calls can be
categorized as follows:
1 (just method name) - importUseCaseModels
2 - importProviderModels
3 to 4 - importGeoLocationModels, importSecurityModel
In the last category, we have the methods for which we need to specify also the constructor
boolean attribute value. Depending on the value of this attribute, we might not need to set a
second parameter for the method call (each method accepts two parameters are most) as this is
not needed in case that we need to store everything in the CDORepository.
Let us provide a small example in order to clarify better things.
Support that the user desires to call importGeoLocationModels and save the result in the
file system and that this user chooses the first way and first option for it. To this end,
the command to provide would be:
exec:java -Deu.paasage.configdir=<path_to_CDOClient_properties_file> -Dexec.args="importGeoLocationModels false input/geopolitical.owl locations.xmi",
where the last two arguments map to the two input parameters of the importGeoLocationModel method.
Now, in case the user desires to store the location model into the CDORepository, he/she does
not need to provide the last argument, as based on the model storage guidelines, the location
model will be stored in a specific place inside the CDO Repository resource folder structure.
Thus, the respective command would be the following:
exec:java -Deu.paasage.configdir=<path_to_CDOClient_properties_file> -Dexec.args="importGeoLocationModels true input/geopolitical.owl",
where now only 3 arguments instead of 4 are provided.
\ No newline at end of file
input/Metric.xmi
View file @ 2dbe3fa0
......@@ -3,10 +3,10 @@
xmi:version="2.0"
xmlns:xmi="http://www.omg.org/XMI"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://www.paasage.eu/camel"
xmlns:metric="http://www.paasage.eu/camel/metric"
xmlns:unit="http://www.paasage.eu/camel/unit"
xsi:schemaLocation="http://www.paasage.eu/camel camel.ecore http://www.paasage.eu/camel/metric camel.ecore#//metric http://www.paasage.eu/camel/unit camel.ecore#//unit"
xmlns:camel="http://www.paasage.eu/2015/06/camel"
xmlns:metric="http://www.paasage.eu/2015/06/camel/metric"
xmlns:unit="http://www.paasage.eu/2015/06/camel/unit"
xsi:schemaLocation="http://www.paasage.eu/2015/06/camel camel.ecore http://www.paasage.eu/2015/06/camel/metric camel.ecore#//metric http://www.paasage.eu/2015/06/camel/unit camel.ecore#//unit"
name="Basic Metric Model">
<metrics
xsi:type="metric:RawMetric"
......
input/providers/CloudLocationModel.xmi
View file @ 2dbe3fa0
......@@ -3,8 +3,8 @@
xmi:version="2.0"
xmlns:xmi="http://www.omg.org/XMI"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:location="http://www.paasage.eu/camel/location"
xsi:schemaLocation="http://www.paasage.eu/camel/location camel.ecore#//location"
xmlns:location="http://www.paasage.eu/2015/06/camel/location"
xsi:schemaLocation="http://www.paasage.eu/2015/06/camel/location camel.ecore#//location"
name="Cloud Location Model">
<cloudLocations
id="Virtual Data Center">
......
pom.xml
View file @ 2dbe3fa0
......@@ -60,12 +60,12 @@
<dependencies>
<dependency>
<groupId>eu.paasage</groupId>
<groupId>org.ow2.paasage</groupId>
<artifactId>camel</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>eu.paasage.mddb.cdo</groupId>
<groupId>org.ow2.paasage.mddb.cdo</groupId>
<artifactId>client</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
......
src/eu/paasage/mddb/model/importer/ModelImporter.java
View file @ 2dbe3fa0
......@@ -57,24 +57,38 @@ import eu.paasage.mddb.model.importer.security.SecurityPopulator;
public class ModelImporter {
private boolean inCdo = false;
private CDOClient cl = null;
private static org.apache.log4j.Logger logger = org.apache.log4j.Logger.getLogger(CDOClient.class);
public ModelImporter(){
}
/* the constructor takes as input a boolean parameter indicating whether we store imported
* or generated information in the CDO repository. In case we do, then the respective
* CDO Repository resource folder structure is created.
*/
public ModelImporter(boolean inCdo){
this.inCdo = inCdo;
cl = new CDOClient();
if (inCdo) createRepositoryStructure();
}
/* This method is used to import geolocations in the form of a location model
* into the CDO Repository or the file system. It takes as input two parameters.
* The first parameter indicates the path to the file where a geolocation ontology
* is situated which is processed in order to create the location model. The second
* parameter is the path of the file to store the location model, in case that
* the inCdo boolean parameter has been set to false. The output of this method indicates
* whether the importing has been successful or not.
*/
public boolean importGeoLocationModels(String path, String outputPath){
LocationGenerator lg = new LocationGenerator(path);
LocationModel lm = lg.getLocationModel();
if (lm != null){
boolean stored = false;
if (inCdo){
boolean created = createCDOFolderStructure(outputPath,null);
if (created) stored = cl.storeModel(lm, outputPath + "/" + outputPath, true);
boolean created = createCDOFolderStructure("locations",null);
if (created) stored = cl.storeModel(lm, "locations/locations", true);
}
else{
stored = cl.exportModel(lm, outputPath);
......@@ -84,6 +98,11 @@ public class ModelImporter {
return false;
}
/* This method just creates a specific CDO resource folder path and sub-path,
* where both paths are given in the form of Strings as input parameters to
* this method. The output indicates whether the creation of the folder path
* and sub-path in the CDO Repository has been successful.
*/
private boolean createCDOFolderStructure(String path,String subPath){
try{
CDOTransaction trans = cl.openTransaction();
......@@ -94,12 +113,16 @@ public class ModelImporter {
return true;
}
catch(Exception e){
System.out.println("Something went wrong while creating Root Folder for Providers in CDO Repository");
e.printStackTrace();
logger.error("Something went wrong while creating Root Folder for Providers in CDO Repository",e);
}
return false;
}
/* This method is used to import one or more provider models into the
* CDO repository, where these models are stored in a specific directory
* whose path in the file system is given as input to this method. The
* output indicates whether the provider models have been successfully stored.
*/
public boolean importProviderModels(String dirPath){
if (inCdo){
File f = new File(dirPath);
......@@ -130,7 +153,7 @@ public class ModelImporter {
trans.commit();
}
catch(Exception e){
e.printStackTrace();
logger.error("Something went wrong while importing provider models",e);
}
trans.close();
//cl.storeModel(model,"/orgs/cps/" + name + "/" + name,true);
......@@ -144,15 +167,20 @@ public class ModelImporter {
return false;
}
/* This method is called in order to fix the locations in the requirement models
* and data centers that have been stored in the CDO Repository according to
* a particular location model which is given as input along with the respective
* transaction under which this fixing takes place.
*/
private void fixLocationModel(LocationModel model, CDOTransaction trans){
for (Country country: model.getCountries()){
String name = country.getName();
System.out.println("Processing country: " + name);
logger.debug("Processing country: " + name);
Country corCountry = trans.createQuery("hql", "select c from Country c where c.name like '%" + name + "%' or '" + name + "' member of c.alternativeNames").getResult(Country.class).get(0);
Collection<Setting> references = EcoreUtil.UsageCrossReferencer.find(country, country.eResource());
for (Setting setting : references) {
EObject source = setting.getEObject();
System.out.println("Source is: " + source);
logger.debug("Source is: " + source);
if (source instanceof LocationRequirement){
LocationRequirement lr = (LocationRequirement)source;
lr.getLocations().remove(country);
......@@ -166,12 +194,12 @@ public class ModelImporter {
}
for (GeographicalRegion region: model.getRegions()){
String name = region.getName();
System.out.println("Processing region: " + name);
logger.debug("Processing region: " + name);
GeographicalRegion corRegion = trans.createQuery("hql", "select r from GeographicalRegion r where r.name='" + name + "' or '" + name + "' member of r.alternativeNames").getResult(GeographicalRegion.class).get(0);
Collection<Setting> references = EcoreUtil.UsageCrossReferencer.find(region, region.eResource());
for (Setting setting : references) {
EObject source = setting.getEObject();
System.out.println("Source is: " + source);
logger.debug("Source is: " + source);
if (source instanceof LocationRequirement){
LocationRequirement lr = (LocationRequirement)source;
lr.getLocations().remove(region);
......@@ -185,6 +213,10 @@ public class ModelImporter {
}
}
/* This method is used for processing and storing in the CDO Repository a use case
* model that is given as input. The output indicates whether the storage has been
* successful.
*/
private boolean processUseCaseModel(EObject model){
if (model instanceof CamelModel){
CamelModel cm = (CamelModel)model;
......@@ -249,7 +281,7 @@ public class ModelImporter {
trans.close();
}
catch(Exception e){
e.printStackTrace();
logger.error("Something went wrong while importing use case model / user-specific part",e);
}
if (stored){
try{
......@@ -263,7 +295,7 @@ public class ModelImporter {
trans.close();
}
catch(Exception e){
e.printStackTrace();
logger.error("Something went wrong while fixing cross-references between user and organisation models according to the model storage guidelines",e);
}
cl.exportModel(orgPath + "/" + orgName, OrganisationModel.class, "output/OrgModel.xmi");
cl.exportModel(userPath + "/" + userEmail, CamelModel.class, "output/UserModel.xmi");
......@@ -273,6 +305,13 @@ public class ModelImporter {
return false;
}
/* This method is used to import use case models. It is assumed that these models have been
* stored in a specific jar file which is inside the dependencies of this code. For
* the current time being, only the Scalarm use case model is imported but the code doing
* that is quite generic to accommodate also other use case models. It is expected that
* the signature of the method will be modified to include the resource paths to the use case
* models to import. The method returns as output the result of the importing in terms of success.
*/
public boolean importUseCaseModels(){
URL url = cl.getClass().getResource("/Scalarm.xmi");
EObject model = cl.loadModel(url);
......@@ -280,12 +319,21 @@ public class ModelImporter {
return processed;
}
/* This method just creates an empty type model and returns it as output.*/
private EObject createEmptyTypeModel(){
TypeModel tm = TypeFactory.eINSTANCE.createTypeModel();
tm.setName("Camel Type Model");
return tm;
}
/* This method is used to import cloud locations in the location model stored in the
* CDO Repository. This method will be modified and merged into the one of importing
* provider models as cloud locations are specific to a certain cloud provider and
* are not specified in a separate file/model!!!
* The method takes as input the path to the file system where the cloud location model
* is stored and returns as output a boolean value indicating whether the importing
* into the CDO Repository was successful or not.
*/
private boolean importCloudLocations(String path){
boolean ok = false;
LocationModel lm = (LocationModel)cl.loadModel(path);
......@@ -298,7 +346,7 @@ public class ModelImporter {
DataCenter gwdg = trans.createQuery("hql", "select dc from CloudProvider cp, DataCenter dc where cp.name='GWDG' and dc.cloudProvider=cp").getResult(DataCenter.class).get(0);
for(CloudLocation cloc: cloudLocations){
String id = cloc.getId();
System.out.println("Checking CloudLocation: " + id);
logger.debug("Checking CloudLocation: " + id);
CloudLocation cloc2 = LocationFactory.eINSTANCE.createCloudLocation();
cloc2.setId(cloc.getId());
cloc2.setIsAssignable(cloc.isIsAssignable());
......@@ -319,12 +367,16 @@ public class ModelImporter {
ok = true;
}
catch(Exception e){
e.printStackTrace();
logger.error("Something went wrong while importing cloud locations",e);
}
trans.close();
return ok;
}
/* This method is used for creating a basic unit model which covers all possible units
* that can be created by the unit meta-model. This basic unit model is finally returned
* as output of this method.
*/
private EObject createUnitModel(){
UnitModel um = UnitFactory.eINSTANCE.createUnitModel();
um.setName("Global Unit Model");
......@@ -413,6 +465,9 @@ public class ModelImporter {
return um;
}
/* This method is used to create the CDO Repository resource folder structure
* according to the model storage guidelines specified in the content of PaaSage WP2
*/
public void createRepositoryStructure(){
CDOClient cl = new CDOClient();
CDOTransaction trans = cl.openTransaction();
......@@ -437,30 +492,113 @@ public class ModelImporter {
trans.commit();
}
catch(Exception e){
e.printStackTrace();
logger.error("Something went wrong while creating CDO Repository Structure",e);
}
cl.closeTransaction(trans);
cl.closeSession();
}
public boolean importSecurityModel(String path,String resourcePath){
return SecurityPopulator.populate(path, cl, resourcePath);
/* This method is used to generate a basic security model through processing a specific
* file whose path in the file system is provided as input to this method. The method takes
* also a second parameter as input which should be set when the security model generated
* has to be stored in the file system, thus pointing to the file path where the
* model will be stored. The method returns a boolean value indicating whether the
* importing in the CDO Repository or file system has been successful or not.
*/
public boolean importSecurityModel(String path, String resourcePath){
if (inCdo) return SecurityPopulator.populate(path, cl, "sec/security", inCdo);
else return SecurityPopulator.populate(path, cl, resourcePath, inCdo);
}
/* This method should be called when the importing of all types of models has been finished
* as it is related to the closing of the CDOSession established through the internally
* exploited CDOClient.
*/
public void finish(){
cl.closeSession();
}
public static void main(String[] args){
ModelImporter mi = new ModelImporter(true);
mi.createRepositoryStructure();
//mi.importLocationModels("input/geopolitical.owl", "output/locations.xmi");
mi.importGeoLocationModels("input/geopolitical.owl", "locations");
mi.importProviderModels("input/providers");
mi.importCloudLocations("input/providers/CloudLocationModel.xmi");
mi.importSecurityModel("input/CSA_CCM_v3.0.1.xlsx","sec/security");
mi.importUseCaseModels();
mi.finish();
int status = 0;
if (args.length == 0){
ModelImporter mi = new ModelImporter(true);
mi.importGeoLocationModels("input/geopolitical.owl", "locations");
mi.importProviderModels("input/providers");
mi.importCloudLocations("input/providers/CloudLocationModel.xmi");
mi.importSecurityModel("input/CSA_CCM_v3.0.1.xlsx","sec/security");
mi.importUseCaseModels();
mi.finish();
}
else{
String method = args[0];
ModelImporter mi = null;
if (method.equals("importUseCaseModels")){
mi = new ModelImporter(true);
mi.importUseCaseModels();
status = 2;
}
else if (method.equals("importProviderModels") || method.equals("importCloudLocations")){
mi = new ModelImporter(true);
if (args.length > 1){
if (method.equals("importProviderModels"))
mi.importProviderModels(args[1]);
else if (method.equals("importCloudLocations")){
mi.importCloudLocations(args[1]);
}
status = 2;
if (args.length > 2){
status = 1;
}
}
else{
status = 0;
}
}
else if (method.equalsIgnoreCase("importGeoLocationModels") || method.equals("importSecurityModel")){
boolean inCdo = Boolean.parseBoolean(args[1]);
mi = new ModelImporter(inCdo);
if (args.length >= 3){
if (inCdo){
if (method.equals("importGeoLocationModels"))
mi.importGeoLocationModels(args[2], null);
else if (method.equals("importSecurityModel"))
mi.importSecurityModel(args[2], null);
status = 2;
}
else{
if (args.length >= 4){
if (method.equals("importGeoLocationModels"))
mi.importGeoLocationModels(args[2], args[3]);
else if (method.equals("importSecurityModel"))
mi.importSecurityModel(args[2], args[3]);
status = 2;
}
else{
status = 0;
}
}
if (args.length > 4){
status = 1;
}
}
else{
status = 0;
}
}
else{
logger.info("No method with the respective name: " + method + " exists or it is not readable to call it in a command line manner");
status = 2;
}
if (status == 1){
logger.info("More input arguments than needed have been provided. The rest will be ignored");
}
else if (status == 0){
logger.error("Less input arguments than needed have been provided for method: " + method);
}
else{
mi.finish();
}
}
System.exit(0);
}
}
src/eu/paasage/mddb/model/importer/security/SecurityPopulator.java
View file @ 2dbe3fa0
......@@ -68,7 +68,7 @@ public class SecurityPopulator {
return id;
}
public static boolean populate(String path, CDOClient cl, String resourcePath){
public static boolean populate(String path, CDOClient cl, String resourcePath, boolean inCdo){
boolean ok = false;
int secControlNum = 0;
SecurityModel sm = null;
......@@ -156,8 +156,11 @@ public class SecurityPopulator {
}
System.out.println("Number of Security Controls is: " + secControlNum + " number of top domains is: " + secDomIds.size());
//secModels.add(sm);
cl.exportModel(sm, "output/CCM.xmi");
ok = cl.storeModel(sm, resourcePath, true);
//cl.exportModel(sm, "output/CCM.xmi");
if (inCdo)
ok = cl.storeModel(sm, resourcePath, true);
else
ok = cl.exportModel(sm, resourcePath);
return ok;
}
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment