collective.jsonmigrator’s documentation!

The purpose of the collective.jsonmigrator package is to provide a set of blueprints that help you to migrate content into Plone. (blueprint is an extension to collective.transmogrifier).

It provides a list of blueprints built around collective.jsonmigrator.jsonsource with the purpose of providing flexible infrastructure to do migrations in Plone.

And just incase you forgot, migration is a bitch… so have fun :P

collective.jsonmigrator.jsonsource

Read JSON files and insert them into transmogrifier pipeline.

Parameters

path (required):
 

Path to directory containing JSON files (look in example below).

Also possible to specify in some.package:path/to/json/directory way.

Example

Configuration:

[transmogrifier]
pipeline =
    source

[source]
blueprint = collective.jsonmigrator.jsonsource
path = some.package:/path/to/json/dir

JSON files structure:

some.package:/path/to/json/dir
    |-> 0/
        |-> 1.json
        |-> 2.json
        ...
        |-> 999.json
    |-> 1/
        |-> 1000.json
        |-> 1001.json
        ...

JSON file:

{
    "_path": "/Plone/front-page",
    "_type": "Document",
    ...
}

collective.jsonmigrator.skipitems

Skip first N item in pipeline.

Development blueprint. Useful when you are processing big data pipelines and you know that the first N items are already migrated.

Parameters

first (required):
 define number of items from the beginning of data pipeline to skip.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    skipitems

...

[skipitems]
blueprint = collective.jsonmigrator.skipitems
first = 10000

collective.jsonmigrator.partialcommit

Used to commit after some items have been processed.

Parameters

every (default:100): Define number of items after which commit (writing to ZODB) will happen.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    commit

...

[commit]
blueprint = collective.jsonmigrator.partialcommit
every = 500

collective.jsonmigrator.statistics

TODO:

need to fix statistic blueprint so it doesn’t depend on other blueprints to report and add statistic data.

Also reporting should not only be written to stdout.

Parameters

Example

collective.jsonmigrator.workflowhistory

Update the workflow history of an object.

Parameters

No parameters.

Expected data in pipeline:

  • _path: path to object on which we want to change workflow history.
  • _workflow_history: workflow history to be applied to object resolved above.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    workflowhistory

...

[workflowhistory]
blueprint = collective.jsonmigrator.workflowhistory

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_workflow_history": {
        "plone_workflow": [
            {
                "action": null,
                "review_state": "visible",
                "comments": "",
                "actor": "admin",
                "time": "2010/09/15 02:19:57.932 GMT+2"
            }
        ]
    },
}

collective.jsonmigrator.mimetype

Sometimes we need to fix/change the mimetype of migrated objects.

Parameters

No parameters.

Expected data in pipeline:

  • _path: path to object on which we want to change mimetype.
  • _content_type: mimetype to be applied to object resolved above.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    mimetype

...

[mimetype]
blueprint = collective.jsonmigrator.mimetype

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_content_type": "text/html",
}

collective.jsonmigrator.properties

Update properties of an object.

Configuration options

No specific blueprint parameters.

Expected data structure in pipeline:

  • _path: path to object on which we want to change properties.

  • _properties: properties to be applied to object resolved above.

    properties passed in this data field (as shown in example) is a list of 3-item lists.:

    [
    [
        <property-name>,
        <property-value>,
        <property-type>
    ],
    [
        <property2-name>,
        <property2-value>,
        <property2-type>
    ],
    ...
]

    <property-type> is set of types which you can select through the ZMI when you edit/add a property.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    properties

...

[properties]
blueprint = collective.jsonmigrator.properties

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_properties": [
        [
            "title",
            "Welcome to Plone",
            "string"
        ]
    ],
}

collective.jsonmigrator.permission_mapping

Update permissions of an object.

Parameters

No parameters.

Expected data in pipeline:

  • _path: path to object on which we want to change permissions.
  • _permission_mapping: permissions to be applied to object resolved above.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    permission_mapping

...

[mimetype]
blueprint = collective.jsonmigrator.permission_mapping

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_permission_mapping": {
        "Modify portal content": {
            "acquire": false,
            "roles": [
                "Manager",
                "Owner"
            ]
        },
        "Access contents information": {
            "acquire": true,
            "roles": [
                "Anonymous",
                "Manager",
                "Reviewer"
            ]
        },
        "View": {
            "acquire": true,
            "roles": [
                "Anonymous",
                "Manager",
                "Reviewer"
            ]
        }
    },
}

collective.jsonmigrator.owner

Update owner of an object.

Parameters

No parameters.

Expected data in pipeline:

  • _path: path to object on which we want to change properties.
  • _owner: properties to be applied to object resolved above.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    owner

...

[owner]
blueprint = collective.jsonmigrator.owner

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_owner": [
        1,
        "admin"
    ],
}

collective.jsonmigrator.local_roles

Update local roles of an object.

Parameters

No parameters.

  • _path: path to object on which we want to change local roles.
  • _ac_local_roles: local roles to be applied to object resolved above.

Example

Configuration:

[transmogrifier]
pipeline =
    source
    local_roles

...

[local_roles]
blueprint = collective.jsonmigrator.local_roles

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_ac_local_roles": {
        "admin": [
            "Owner"
        ]
    },
}

collective.jsonmigrator.datafields

Update data/blob fields of an object.

TODO:missing base path (maybe even passed somehow from source blueprint)
TODO:only update if needed

Configuration options

No specific blueprint parameters.

Expected data structure in pipeline:

  • _path: path to object on which we want to change local roles.
  • _datafield_<field>: field which needs to store data

Example

This example will try to store content of 0/1.json-file-1 into the attachment field of the /Plone/index_html object.

Configuration:

[transmogrifier]
pipeline =
    source
    datafields

...

[datafields]
blueprint = collective.jsonmigrator.datafields

Data in pipeline:

{
    "_path": "/Plone/index_html",
    "_datafield_attachment": "0/1.json-file-1",
}