There is a new data model Wall in the new subject Zero Emission Buildings.
Thanks to the contributors of Mitsubishi electric for the work and for being the pioneers of the new testing process that improves previous one.
In the new testing process, 4th option in the tools menu, now it is available a new test that checks if the example-normalized.jsonld is a valid NGSI LD file.
This process helps contributors to debug their data models before submit them officially (where there will be new tests before final approval)
The source code for the test is available at the repo.
Remember that if you want to improve / create a new test, just create a PR on the repo.
In the new testing process, 4th option in the tools menu, now it is available a new test that checks if the example-normalized.json is a valid NGSIv2 file.
This process helps contributors to debug their data models before submit them officially (where there will be new tests before final approval)
The source code for the test is available at the repo.
Remember that if you want to improve / create a new test, just create a PR on the repo.
When you want to contribute a new data model (or an improvement in an existing one) you need to pass a test.
The current process (3rd option in tools menu) keeps on working as it was.
But we have drafted a new method because
– We need to be more explicit about the tests passed and the errors
– We need to improve the performance
So you can check the new method in the 4th option of the Tools menu
Besides this, the tests are very modular so if you are a python programmer you can use them in your own system because the code is being released or indeed you can write new tests that would be included in the official site. Make a PR on the data-models repo and we will add it eventually. Check this post.
Current test process for new and extended data models
In order to approve a new data model a test needs to be passed. It cold be accessed in the 3rd option in the tools menu at the front page:
Pro: it is currently working
Con: It is mostly created in a single file for testing and error messages are not very explicit about the errors detected
The new process
1) Every test is an independent file:
2) To test the new data model it copies to local the files and then run the tests, which is quicker.
What can you do with basic knowledge of python (or with a good AI service)
Here you can see the current files available in the github repository data-models subdirectory test_data_model
Instructions
This directory contains the decentralized method to test new and existing data models
The file master_tests.py executes all the files in the tests directory as long as they are included in this line of code
test_files = ["test_valid_json", "test_file_exists", "test_schema_descriptions", "test_schema_metadata", "test_duplicated_attributes"]
so if you create a new test you need to extend this line with your file. Bear in mind these points
The file master_tests.py is invoked this way 'python3 master_tests.py <repo_url_or_local_path> <only_report_errors>' . It expects to have all the tests in the subdirectory tests (like in the repo)
What can be contributed by you. Lots of tests. Just a few
NOTE: We did yesterday 17-9 the changes. Unfortunately we made a mistake and now we have to revert all these changes, do it again properly and push. this Friday will be ready if not earlier.
NOTE2: It is already updated. Its Wednesday 15:30. Hopefully this time we made no errors.
The single-source-of-truth of the data models is the json schema (file schema.json). This json schema has a tag ‘$schema’ indicating the meta schema the schema is compliant with.
Now all data models have been updated to the last one “https://json-schema.org/draft/2020-12/schema”
Therefore some errors provided by validators due to the obsolete previous value have been removed.
Thanks to the user Elliopardad in GitHub for its contribution and to the community of json schema for its support.
As we announce earlier we are one of the project listed in its global landscape of projects.
The directory /code/ (see image with one example) in every data model has now a new draft export the pydantic export.
Pydantic is a Python library that provides data validation and settings management using Python type annotations, allowing you to define data models that enforce type constraints and validate data automatically.
Now in most (if not all) data models you have such export to use it freely. Mind that is a first version and errors could happen (It is welcomed if you find any error or just make a suggestion)
The Smart Data Models (SDM) initiative, led by FIWARE Foundation in collaboration with IUDX, TM Forum, and OASC, has firmly established JSON Schema as the core component and single source of truth for creating exports in YAML, SQL, and soon RDF. This strategic move aligns the SDM initiative with the growing JSON schema community, enabling a wider adoption of this powerful data modeling standard.
The SDM initiative is an open collaboration aiming to promote the adoption of a reference architecture and compatible common data models across various sectors, starting with Smart Cities. By leveraging JSON schema as the foundation, the initiative ensures that the data models developed are not only technically robust but also interoperable with a wide range of semantic and linked data initiatives.
“The adoption of JSON schema as the core component of the Smart Data Models initiative is a significant step forward in our mission to enable interoperable smart solutions,” said Alberto Abella (Data Modeling Expert, FIWARE Foundation). “This collaboration with the JSON schema community will further strengthen the initiative and drive the widespread adoption of these common data models.”
In addition to the JSON schema-based data models, the SDM initiative also creates comprehensive specifications in eight languages, including English, French, German, Spanish, Italian, Korean, Chinese, and Japanese. This multilingual approach ensures that the data models are accessible and usable by a global audience, fostering international collaboration and knowledge sharing.
“The alignment of the Smart Data Models initiative with the JSON Schema community is a testament to the power and versatility of this data modeling standard,” said Benjamin Granados (Community Development Senior Manager – Open Technologies, JSON Schema Community, Postman). “We are excited to work closely with the SDM team to further enhance the adoption and integration of JSON schema across various smart applications and services.”
The Smart Data Models initiative welcomes contributions from the public. The data models are licensed under a royalty-free, open-source model, permitting free use, modification, and sharing. This collaborative approach fosters innovation and the creation of interoperable smart solutions, which can be replicated and scaled across various sectors and regions.
For more information about the Smart Data Models initiative and its adoption of JSON schema, please visit the official website at https://smartdatamodels.org or follow the initiative on X @smartdatamodels or in Linkedin.
Due to the new configuration of files of the package pysmartdatamodels it will be no longer required to use the from clause (initially)
Therefore now to import the package in python it will be simply
import pysmartdatamodels as sdm
Accordingly the examples of code in all data models are being changed, including a comment on this version change.
This updated of the examples o code will be announced soon.
Some of the users of the Smart Data Models are public entities. Those entities are willing to use Smart Data Models in the provisioning of their IT systems.
They can do it because SDM are open licensed models not depending on any software maker but in public standards and the license of the data models allows them to customize the models and to share the modifications with only attributing the authors.
Here you can see and comment a draft document with some examples of the technical clauses for public tenders (currently only in Spanish and English)
It is open for comments and suggestions
This document arised as a consequence of a webinar held past May 16th with the Spanish Network of Smart Cities (RECI).
