The data models available for python developers. pysmartdatamodels 0.5.40 published. Beta version.

Now you can find in pypi.org the python package pysmartdatamodels with 13 functions for the integrators of the data models (more than 800) in external systems and applications. It is a beta version. There is a function, update_data() that whenever is run, it updates the data models to the last version (including adding the new data models). The code is available at the utils directory.
This python package includes all the data models and several functions (listed below) to use in your developments.

If you want to be updated on this package you can join this mailing list (Announcements are sent only when something relevant happens). We love to get your feedback at info@smartdatamodels.org

There are several online tools to manage and to create the data models, generate examples or to adapt to existing ontologies. See the tools menu option at the home site.

Functions available include:

1- List all data models. Function list_all_datamodels()
2- List all subjects. Function list_all_subjects()
3- List the data models of a subject. Function datamodels_subject(subject)
4- List description of an attribute. Function description_attribute(subject, datamodel, attribute)
5- List data-type of an attribute. Function datatype_attribute(subject, datamodel, attribute)
6- Give reference model for an attribute. Function model_attribute(subject, datamodel, attribute)
7- Give reference units for an attribute. Function attributes_datamodel(subject, datamodel)
8- List the attributes of a data model. Function attributes_datamodel(subject, datamodel)
9- List the NGSI type (Property, Relationship or Geoproperty) of the attribute. Function ngsi_datatype_attribute(subject, datamodel, attribute)
10- Print a list of data models attributes separated by a separator. Function print_datamodel(subject, datamodel, separator, meta_attributes)
11- Returns the link to the repository of a subject. Function subject_repolink(subject)
12- Returns the links to the repositories of a data model name. Function datamodel_repolink(datamodel)
13- Update the official data model list or the database of attributes from the source. Function update_data()

Roadmap

1.- Create a proper documentation

2.- Function to allow submission of improvements (i.e. missing recommended units or model) and comments to the different data models. Currently, you can do it by searching for your data model here
https://smartdatamodels.org/index.php/list-of-data-models-3/ visiting the github repo and making your PR or raising your issues there.
3.- Function to submit a new data model to an incubation repository. Currently, this is done manually incubated repository. By filling this form you are granted to contribute with new data models.
4.- Include new functions like search for the subject of a data model or other that you can suggest to us at info@smartdatamodels.org

### some example code
from pysmartdatamodels import pysmartdatamodels as sdm
subject = "dataModel.Weather"
dataModel = "WeatherForecast"
attribute = "precipitation"
print(sdm.list_all_datamodels())
print(sdm.list_all_subjects())
print(sdm.datamodels_subject("dataModel.Weather"))
print(sdm.description_attribute(subject, dataModel, attribute))
print(sdm.datatype_attribute(subject, dataModel, attribute))
print(sdm.model_attribute(subject, dataModel, attribute))
print(sdm.units_attribute(subject, dataModel, attribute))
print(sdm.attributes_datamodel(subject, dataModel))
print(sdm.subject_repolink(subject))
print(sdm.datamodel_repolink(dataModel))
print(sdm.print_datamodel(subject, dataModel, ",", ["property", "type", "dataModel", "repoName", "description", "typeNGSI", "modelTags", "format", "units", "model"]))
sdm.update_data()

SQL export available for Postgresql

There is a new file ‘schema.sql‘ in all the directories of the data models. It is a SQL script for PostgreSQL.

The script creates the structure of a relational table containing the attributes defined in the data model. It also creates the data types for those attributes with an enumeration of values.

For those attributes being arrays or objects, it creates a JSON attribute (allowed in PostgreSQL).

If you need additional features in this export please report them to info@smartdatamodels.org.

See an example.

Postgresql elephant

Draft of a python package available

Now we have a draft version of a python package to integrate the smart data models with your developments. It is a beta version so you can expect some issues when using it. We will be glad if you report it at info@smartdatamodels.org or suggest new features. Thanks to Anthony Uphof for his contributions (only a few of them are in this draft, next version will include them)

To install

pip install -i test.pypi.org/simple/ pysmartdatamodels

The functions included are:

1- List all data models. Function list_all_datamodels()

2- List all subjects. Function list_all_subjects()

3- List the data models of a subject. Function datamodels_subject(subject)

4- List description of an attribute. Function description_attribute(subject, datamodel, attribute)

5- List data-type of an attribute. Function datatype_attribute(subject, datamodel, attribute)

6- Give reference model for an attribute. Function model_attribute(subject, datamodel, attribute)

7- Give reference units for an attribute. Function attributes_datamodel(subject, datamodel)

8- List the attributes of a data model. Function attributes_datamodel(subject, datamodel)

9- List the NGSI type (Property, Relationship or Geoproperty) of the attribute. Function ngsi_datatype_attribute(subject, datamodel, attribute)

11- Print a list of data models attributes separated by a separator. Function print_datamodel(subject, datamodel, separator, meta_attributes)

12- Update the official data model list or the database of attributes from the source. Function update_data()

New Observation data model at the new subject SDMX

SDMX is a standard for the codification of statistical information. Retrieving these types of data and inserting them in systems based on JSON/JSON-LD is a tough task performed by the project Interstat.

Thanks to its collaboration we have a new subject, dataModel.SDMX where these data models for mapping this standard will be located. this subject belongs to the Cross-Sector domain.

The first one is the Observation data model.

  • Observation. A single observation in the SDMX Cube, may have one or more associated measured values

Sdmx logo

New database of data models’ versions

In the main menu, it has been extended the submenu of list data models with a complete database of all versions of the data models.

It includes not only the data model, subject, and version but also the data directly linked to the raw version of the data model.

You can find it at Home -> List of data models -> List of data models versions.

Semantic-versioning

Release source code for some of the services. For python developers.

In the tools option of the main menu, there are a few services to help you to use, create and debug your data models.

Although working, some of these services are limited. We are releasing the code for some of the services to allow you to contribute with improvments.

It appears by the end of the page with texts like this:

“source code of this service if you want to improve it.”

Source code has an hyperlink pointing to the specific script in the utils directory.

They are created in python and connected to the WP forms in each option

Specifically

List of Adopters (use cases) and contributors available at Community menu

On the front page under the Community menu, there are two new entries
Adopters: Lists the use cases documented in the different data models. It includes searchable facilities. It contains these fields:  adopter, description, mail, organization, project, comments, start date, subject and data model.

If you want to be listed, just make a PR on the file ADOPTERS.yaml of the data model folder. (The PR has to include an example ‘payload’ on how you use it)

Contributors: Lists the people. It contains these fields:  name, surname, mail, organization, project, comments, year, and subject.

If you want to be listed you have to have contributed to any of the data models of the subject and then make a PR on the CONTRIBUTORS.yaml at the root of the subject.

In both cases the attributes are not mandatory so they can be empty.

New information customized for the different profiles

In the documentation menu (Home -> documentation -> Basic info for:), there are now 4 new options to provide you with the basic information depending on your profile

  • User. For those visitors with limited knowledge about what are the Smart Data Models
  • Contributor. For those visitors willing to extend or to contribute with new data models
  • Developer. For those visitors willing to integrate the data models with other tools or initiatives
  • Researcher. For those visitors whose aim is to understand what is agile standardization and how it is implemented at the Smart Data Models Program

If you have some of these profiles and you miss some information please let us know

Updating all specifications to make it easier to be updated

Initially, you will see few changes in the specifications of all data models,(it is in progress because it will last around 2 days to get it completed)

– The inclusion of the model for those attributes having it.

– Including the data type (when there is only one) for the attributes. ”

– A footer with some useful links, etc

Smart Data Models +++ Contribution Manual +++ About

but there is another hidden relevant change. The specifications are now divided into sections by these tags:

<!– section name –>

<!– /section name –>

It makes easier to update part of the specification without disrupting the rest of the content, and in most markdown viewers it is unnoticed. Besides this, it also allows the extension of the specification (if needed in a coming future) with new sections.

And, of course, everything can be done automatically. Thus we can keep being agile according to our principles.

Therefore, if you see that the specifications are presenting an update don’t worry, Initially, it is just the format and this new trick.

And welcome to the Chinese translation which is using this new format already.