Category: Governance

New version of pysmartdatamodels package 0.7.2

  By

The changes in this new version are:

  • Including a new function to find the subject based on the data model name (In example when in a payload only is available the entity type)
  • for this function to be shown it has to be included a function to load the content open_jsonref
  • Extending the README.md

It also has an updated version of all data models (but you can get this also by running the function sdm.update_data() with the old versions)

The source code for the new version 0.7.2 is here at the data-models repository

 

 

New version of the spreadsheet for drafting data models

  By

You can see in the main page an image like the one below.

It takes you to a google spreadsheet where you can draft simple data models even without knowledge of json schema.

Json schema is the format that stores the master description of the data models in the Smart Data Models.

Now there are two additional tabs, ADOPTERS.yaml and notes.yaml which helps you to fill those files which are mandatory to be present but empty if you do not want to disclose your use case or you do not need to add additional explanations about the submitted data model.

Note: if you want to use the spreadsheet you need to:

– Create a local copy in your google account

– Allow the scripts to be executed

– Fill the yellow cells and click on the buttons

Feedback is always welcomed at info@smartdatamodels.org

New test service for contributors. Strongly recommended before contributing a data model.

  By

There is a new service for simplifying the testing of a contribution to a data model.

The new service will check it, just by providing the link to the root of the folder in an publicly accessible repository

  • That the schema.json is
    • Present in the right position
    • It is a valid json and it has the right structure (types and descriptions)
    • It has the right metadata, and the required section
    • External references are valid
    • The descriptions are included
  • That the examples are:
    • Present in the right position
    • Key values are validated by the schema
  • Notes.yaml and ADOPTERS.yaml are present in the right position

all the files (mandatory ones like schema and examples) and the optional (notes.yaml, ADOPTERS.yaml)

There is a specific page for this test in the contribution manual

Extended context.jsonld in all data models

  By

Every subject — a group of data models — has a file named contex.jsonld at their root.

This file contains the @context with automatically generated IRI for the terms used in the data model.

The Entity types are also available in this file with their long IRI.  In the pointed example are the entries for “SeaConditions”, “WeatherAlert”, “WeatherForecast”, and “WeatherObserved”

Remember that if you want to use other IRI coming, for example, from existing ontologies you can use the service on the Home -> Tools -> Mapper @context with external ontologies

The data models available for python developers. pysmartdatamodels 0.5.40 published. Beta version.

  By , , , , , , , , , , , , ,

Now you can find in pypi.org the python package pysmartdatamodels with 13 functions for the integrators of the data models (more than 800) in external systems and applications. It is a beta version. There is a function, update_data() that whenever is run, it updates the data models to the last version (including adding the new data models). The code is available at the utils directory.
This python package includes all the data models and several functions (listed below) to use in your developments.

If you want to be updated on this package you can join this mailing list (Announcements are sent only when something relevant happens). We love to get your feedback at info@smartdatamodels.org

There are several online tools to manage and to create the data models, generate examples or to adapt to existing ontologies. See the tools menu option at the home site.

Functions available include:

1- List all data models. Function list_all_datamodels()
2- List all subjects. Function list_all_subjects()
3- List the data models of a subject. Function datamodels_subject(subject)
4- List description of an attribute. Function description_attribute(subject, datamodel, attribute)
5- List data-type of an attribute. Function datatype_attribute(subject, datamodel, attribute)
6- Give reference model for an attribute. Function model_attribute(subject, datamodel, attribute)
7- Give reference units for an attribute. Function attributes_datamodel(subject, datamodel)
8- List the attributes of a data model. Function attributes_datamodel(subject, datamodel)
9- List the NGSI type (Property, Relationship or Geoproperty) of the attribute. Function ngsi_datatype_attribute(subject, datamodel, attribute)
10- Print a list of data models attributes separated by a separator. Function print_datamodel(subject, datamodel, separator, meta_attributes)
11- Returns the link to the repository of a subject. Function subject_repolink(subject)
12- Returns the links to the repositories of a data model name. Function datamodel_repolink(datamodel)
13- Update the official data model list or the database of attributes from the source. Function update_data()

Roadmap

1.- Create a proper documentation

2.- Function to allow submission of improvements (i.e. missing recommended units or model) and comments to the different data models. Currently, you can do it by searching for your data model here
https://smartdatamodels.org/index.php/list-of-data-models-3/ visiting the github repo and making your PR or raising your issues there.
3.- Function to submit a new data model to an incubation repository. Currently, this is done manually incubated repository. By filling this form you are granted to contribute with new data models.
4.- Include new functions like search for the subject of a data model or other that you can suggest to us at info@smartdatamodels.org

### some example code
from pysmartdatamodels import pysmartdatamodels as sdm
subject = "dataModel.Weather"
dataModel = "WeatherForecast"
attribute = "precipitation"
print(sdm.list_all_datamodels())
print(sdm.list_all_subjects())
print(sdm.datamodels_subject("dataModel.Weather"))
print(sdm.description_attribute(subject, dataModel, attribute))
print(sdm.datatype_attribute(subject, dataModel, attribute))
print(sdm.model_attribute(subject, dataModel, attribute))
print(sdm.units_attribute(subject, dataModel, attribute))
print(sdm.attributes_datamodel(subject, dataModel))
print(sdm.subject_repolink(subject))
print(sdm.datamodel_repolink(dataModel))
print(sdm.print_datamodel(subject, dataModel, ",", ["property", "type", "dataModel", "repoName", "description", "typeNGSI", "modelTags", "format", "units", "model"]))
sdm.update_data()

Release the script for subjects’ context consolidation.

  By

The utils directory at the data models compiles some scripts we use internally.

Now you have available a script for consolidating several @contexts from several subjects.

It is the script called by the main menu options Home->tools -> Subjects’ @context merger

the help of the script

# This file takes several @contexts and merges them creating two files
# context.jsonld with the elements successfully merged
# and conflicts.json that shows those attributes clashing.
# clashing in conflicts file has to be solved manually
# INPUT PARAMETERS (merge_subjects_context.json, outputToFile)
# parameter merge_subjects_context.json
# it is the full path to a file where the path to the @contexts will be located
# See an example of the file below
# If not provided it merges all subjects in the smart data models program
# {
# “dataModel.Weather”: “https://raw.githubusercontent.com/smart-data-models/dataModel.Weather/master/context.jsonld”,
# “dataModel.Battery”: “https://raw.githubusercontent.com/smart-data-models/dataModel.Battery/master/context.jsonld”,
# “dataModel.Building”: “https://raw.githubusercontent.com/smart-data-models/dataModel.Building/master/context.jsonld”,
# “dataModel.Device”: “https://raw.githubusercontent.com/smart-data-models/dataModel.Device/master/context.jsonld”,
# }
# parameter ouputToFile
# when True it outputs two files conflicts.json and context.jsonld
# conflicts.json stores the conflict in the name of attributes (to be solved manually)
# context.jsonld has the attribute name and the Smart Data Models local IRI

MAS: Manifesto for Agile Standardization

  By , , , , , , , , , , ,

The Manifesto for Agile Standardization (MAS) describes the 7 principles that we apply to the Smart Data Models program.

0. Don't just standardize, be agile and standardize
1. Do not reinvent the wheel
2. Normalize real cases
3. Be open
4. Don't be overly specific
5. Flat not Deep
6. Sustainability is key

If you want to read the agile standardization manifesto’s complete explanation a one-page document is located at the root of our data models repository.

 

Export of the full database of attributes

  By

At home -> Search -> json export of attributes database of smart data models  is the full database of attributes (more than 18000), see the statistics page as an array of JSON objects.

Fields for each attribute

_id: identifier of the item

property: the name of the attribute

dataModel: the data model this attribute is present

repoName: the subject this data model belongs to

description: the description of the attribute

typeNGSI: Whether it is a property, Geoproperty, or relationship

modelTags: inherited from the data model tags

license: link to the license for the data model

schemaVersion: version of the data model

type: data type

model: when available the reference model for the attribute

units: when available the recommended units for the attribute

format: either date, or time, or date-time, or URI, etc the format of the attribute