Data Modeling – What is Data Modeling?

Data Models, Data Modeling, What is data Modeling, logical Model, Conceptual Model, Physical Model

Data Models

Data modeling is the documenting of data relationships, characteristics, and standards based on its intended use of the data.   Data modeling techniques and tools capture and translate complex system designs into easily understood representations of the data creating a blueprint and foundation for information technology development and reengineering.

A data model can be thought of as a diagram  that illustrates the relationships between data. Although capturing all the possible relationships in a data model can be very time-intensive, a well-documented models allow stakeholders to identify errors and make changes before any programming code has been written.

Data modelers often use multiple models to view the same data and ensure that all processes, entities, relationships and data flows have been identified.

There are several different approaches to data modeling, including:

Concept Data Model (CDM)

  • The Concept Data Model (CDM) identifies the high level information entities, their relationships, and organized in the Entity Relationship Diagram (ERD).

Logical Data Model (LDM)

  • The Logical Data Model (LDM)  defines detail business information (in business terms) within each of the Concept Data Model and is a refinement of the information entities of the Concept Data Model.   Logical data model are non-RDBMS specific  business definition of tables, fields, and attributes contained within each information entity from which the Physical Data Model (PDM) and Entity Relationship Diagram (ERD) is produced.

Physical Data Model (PDM)

  • The Physical Data Model (PDM)  provides the actual technical details of the model and database object (e.g. table names, field names, etc.) to facilitate creation of accurate detail technical designs and actual database creation.  Physical Data Models are RDBMS specific definition of the logical model used build database, create deployable DDL statements, and to produce the Entity Relationship Diagram (ERD).

Related References

 

Information Management Unit Testing

Information Management Unit Testing, UT, Unit Test

Information Management Unit Testing

 

Information management projects generally have the following development work:

  • Data movement software;
  • Data conversion software;
  • Data cleansing routines;
  • Database development DDL; and
  • Business intelligence and reporting analytical solutions.

Module testing validates that each module’s logic satisfies requirements specified in the requirements specification.

Effective  Practices

  1. Should focus on testing individual modules to ensure that they perform to specification, handle all exceptions as expected, and produce the appropriate alerts to satisfy error handling.
  2. Should be performed in the development environment.
  3. Should be conducted by the software developer who develops the code.
  4. Should validate the module’s logic, adherence to functional requirements and adherence to technical standards.
  5. Should ensure that all module source code has been executed and each conditional logic branch followed.
  6. Test data and test results should be recorded and form part of the release package when the code moves to production.
  7. Should include a code review, which should:
  • Focus on reviewing code and test results to provide additional verification that the code conforms to data movement best practices and security requirement; and
  • Verify that test results confirm that all conditional logic paths were followed and that all error messages were tested properly.

Testing Procedures

  1. Review design specification with designer.
  2. Prepare test plan before coding.
  3. Create test data and document expected test results.
  4. Ensure that test data validates the module’s logic, adherence to functional requirements and adherence to technical standards.
  5. Ensure that test data tests all module source code and each conditional logic branch.
  6. Conduct unit test in personal schema.
  7. Document test results.
  8. Place test data and test results in project documentation repository.
  9. Check code into code repository.
  10. Participate in code readiness review with Lead Developer.
  11. Schedule code review with appropriate team members.
  12. Assign code review roles as follows:
  • Author, the developer who created the code;
  • Reader, a developer who will read the code during the code review—The reader may also be the author; and
  • Scribe, a developer who will take notes.

Code Review Procedures

  1. Validate that code readiness review has been completed.
  2. Read code.
  3. Verify that code and test results conform to data movement best practices.
  4. Verify that all conditional logic paths were followed and that all error messages were tested properly.
  5. Verify that coding security vulnerability issues have been addressed.
  6. Verify that test data and test results have been placed in project documentation repository.
  7. Verify that code has been checked into code repository.
  8. Document action items.

Testing strategies

  1. Unit test data should be created by the developer and should be low volume.
  2. All testing should occur in a developer’s personal schema.

Summary

Unit testing is generally conducted by the developer who develops the code and validates that each module’s logic satisfies requirements specified in the requirements specification.

Useful Tools for Technical Consultants and Developers

Here is a quick list of technical consultant and developer tools (free or open source), which have proven useful over the years to me.  Hopefully, this information will be of use to you has well.

Text and Code Editors

Notepad++

Notepad++, which I have been using for years now, is a free source code editor which supports several programming languages running under the MS Windows environment (governed by GPL License).  I have repeatedly found Notepad++ extraordinarily useful.  Especially, the language features (e.g. Shell, XML, SQL) or needing to work with files and data without the worry of hidden character.

Image and Screenshot Tools

Greenshot

Greenshot is a light-weight screenshot software tool for Windows, which can be very helpful for capturing application screenshots.  Greenshot allows quick creation of screenshots, easy annotation, highlighting and obfuscating using the built-in image editor and sending the screenshot to a file, the clipboard, a printer or as e-mail attachment

PhotoScape

PhotoScape is an easy photo editing software editing, fixing and enhance photos, screenshots, and images.

Useful Open Source Tools for Project Plans

Google – gantterforgoogledrive

This tool can be to create projects and Gantter charts, when you don’t have a project planning tool available. this tools does seam to have some size and complexity limitations.

Microsoft Project Viewer

This is a useful Microsoft Project plan view for those times when you have MS project available.

ProjectLibre OPENPROJ

I have found this tool to be a useful substitute for MS Project and I have not had any issues with size or complexity limitations.  I have also found the export and import capability useful when working on projects teams who do no use a consistent project planning tool.  Additionally, if you are or have been a MS Project user, you should most of the functionality moderately intuitive, at least, I didn’t have any problems adapting.

Diagram Viewers

Microsoft Visio 2016 Viewer

The Microsoft Visio Viewer can be very handy when you need to view Visio diagrams and there are not enough licenses to go around, or you just need to read them and have no need to create or update the diagrams.

Google chrome addin 

if you are a chrome user you may want to consider this alternative.  Especially, if you are a MAc and/or Linux user. 

Where do data models fit in the Software Development Life Cycle (SDLC) Process?

Data Model SDLC Relationship Diagram

Data Model SDLC Relationship Diagram

In the classic Software Development Life Cycle (SDLC) process, Data Models are typically initiated, by model type, at key process steps and are maintained as data model detail is added and refinement occurs.

The Concept Data Model (CDM) is, usually, created in the Planning phase.   However,  creation the Concept Data Model  can slide forwarded or backward,  somewhat , within the System Concept Development, Planning, and Requirements Analysis phases, depending upon  whether the application being modeled is a custom development effort or a modification of a Commercial-Off-The-Shelf (COTS) application.  The CDM is maintained, as necessary, through the remainder of the SDLC process.

The Logical Data Model (LDM) is created in the Requirement Analysis phase and is a refinement of the information entities of the Concept Data Model. The LDM is maintained, as necessary, through the remainder of the SDLC process.

The Physical Data Model (PDM) is created in the Design phase to facilitate creation of accurate detail technical designs and actual database creation. The PDM is maintained, as necessary, through the remainder of the SDLC process.

Related References: