ML Metadata - C2_W3_Lab_1_MLMetadata - Data model

Course Q&A Machine Learning Engineering for Production(MLOps) Machine Learning Data Lifecycle in Production
1

Can anyone explain Data model in plane english.
1 simple easy to understand definition of what does each component do + many examples. ex.: what is an artifact?

This is just not a sufficient enough explanation:

  • ArtifactType describes an artifact’s type and its properties that are stored in the metadata store. You can register these types on-the-fly with the metadata store in code, or you can load them in the store from a serialized format. Once you register a type, its definition is available throughout the lifetime of the store.
  • An Artifact describes a specific instance of an ArtifactType , and its properties that are written to the metadata store.
  • An ExecutionType describes a type of component or step in a workflow, and its runtime parameters.
  • An Execution is a record of a component run or a step in an ML workflow and the runtime parameters. An execution can be thought of as an instance of an ExecutionType . Executions are recorded when you run an ML pipeline or step.
  • An Event is a record of the relationship between artifacts and executions. When an execution happens, events record every artifact that was used by the execution, and every artifact that was produced. These records allow for lineage tracking throughout a workflow. By looking at all events, MLMD knows what executions happened and what artifacts were created as a result. MLMD can then recurse back from any artifact to all of its upstream inputs.
  • A ContextType describes a type of conceptual group of artifacts and executions in a workflow, and its structural properties. For example: projects, pipeline runs, experiments, owners etc.
  • A Context is an instance of a ContextType . It captures the shared information within the group. For example: project name, changelist commit id, experiment annotations etc. It has a user-defined unique name within its ContextType .
  • An Attribution is a record of the relationship between artifacts and contexts.
  • An Association is a record of the relationship between executions and contexts.

Thank you.

2

Hi @sigma,

The following lecture videos go in more depth through the component concepts and their functionalities and for me they were helpful in getting more intuition of the framework:

image

Hope this helps giving you a better understanding of the concepts.
Good luck!

Maarten

2 Likes
3

Thank you. But I need each of the definitions answered on by one. The video still doesn’t even explains what is an artifact. It just immediately start using it without explains what it is. It is probably not an artifact form Indiana Johns or Tomb Raider XD

1.) Explain the following definitions in plane simple english?
2.) Many practical examples of what they can have?
3.) What each of them do?