Bug 280 - Need to be consistent with OAIS clarification of preservation techniques
Summary: Need to be consistent with OAIS clarification of preservation techniques
Status: RESOLVED CHANGE AGREED
Alias: None
Product: Audit and Certification of Trustworthy Digital Repositories
Classification: Unclassified
Component: Section 4 : Digital Object Management (show other bugs)
Version: Sept 2011
Hardware: Yes All
: High Identification of any outdated material
Assignee: David Giaretta (david@giaretta.org)
URL:
Whiteboard:
: 119 (view as bug list)
Depends on:
Blocks: 119 348
  Show dependency tree
 
Reported: 2020-01-13 19:01 UTC by David Giaretta (david@giaretta.org)
Modified: 2023-04-11 14:31 UTC (History)
5 users (show)

See Also:
Organisation of the submitter: MOIMS-DAI
Disposition of the suggested change:
Category of the suggested change: ---
Due date:
Explanation of the reason for the suggested change:
Change needed to be consistent with OAIS clarification of preservation techniques


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Giaretta (david@giaretta.org) 2020-01-13 19:01:35 UTC
The following suggestion:
•	additional preservation techniques have been described explicitly in addition to Migration;

            ==> May be useful to add text to discussion of 4.3.1

OAIS talks about:
 - transformation
 - adding RepInfo
 - handover to another repository

ISO 16363 already has discussion of 
- transformation
- adding Representation.

3.1.2.1  says "The repository shall have an appropriate succession plan, contingency plans, and/or escrow arrangements in place in case the repository ceases to operate or the governing or funding institution substantially changes its scope."

OAIS talks handover of digital objects. This may be necessary even if the archive is not ceasing operation. Maybe one could regard this as "funding institution substantially changes its scope" but that seems a bit of a stretch. So we could make the change:

FROM:
"The repository shall have an appropriate succession plan, contingency plans, and/or escrow arrangements in place in case the repository ceases to operate or the governing or funding institution substantially changes its scope."

TO:
"The repository shall have an appropriate succession plan, contingency plans, and/or escrow arrangements in place in case the repository ceases to operate or the governing or funding institution substantially changes its scope or the repository can no longer commit to long term preservation of specific digital objects."
Comment 1 John Garrett (garrett@his.com) 2020-04-07 06:46:41 UTC
I'm not really a fan of making this change.
I think we should keep 3.1.2.1 focused on major changes to the overall repository.
If there is just a question of not changes to the overall existence of the repository, but we are just handing off individual AIPs or even major collections of AIPs from this OAIS to another OAIS, then there should be a separate place to discuss that.
One place might be in one of the existing metrics in or around 4.3.  Or we could add a new metric if we can't fit it into 4.3.
Comment 2 David Giaretta (david@giaretta.org) 2020-04-20 11:06:37 UTC
The discussion in OAIS about preservation is clearly addressed at individual objects being preserved, not blanket actions on the whole repository.
Therefore it makes sense to me to make the change proposed in http://review.oais.info/show_bug.cgi?id=280#c0
Comment 3 John Garrett (garrett@his.com) 2020-04-30 01:36:34 UTC
(In reply to David Giaretta from comment #2)
> The discussion in OAIS about preservation is clearly addressed at individual
> objects being preserved, not blanket actions on the whole repository.
> Therefore it makes sense to me to make the change proposed in
> http://review.oais.info/show_bug.cgi?id=280#c0

I agree that OAIS addresses individual objects being preserved.

I also agree that we should cover possibility of turning over individual AIPs or AIP Collections within ISO 16363.

I do not think that section 3.1 - GOVERNANCE AND ORGANIZATIONAL VIABILITY of ISO 16363 is the place to address individual AIP issues.

I believe a better place to address individual AIP issues would be in Section 4 - DIGITAL OBJECT MANAGEMENT.  Perhaps in Section 4.3 - PRESERVATION PLANNING or Section 4.4 - AIP PRESERVATION.
Comment 4 David Giaretta (david@giaretta.org) 2020-05-12 12:47:57 UTC
For specific wording, following John's suggestion to add a metric rather then change 3.1.2.1 add:

4.3.5	The repository shall have an appropriate succession plan, contingency plans, and/or escrow arrangements in place in case the repository can no longer commit to long term preservation of specific digital information.

Supporting Text
This is necessary in order to ensure that the objects it had originally committed to preserve will not be lost.

Examples of Ways the Repository Can Demonstrate It Is Meeting This Requirement
Written and credible succession and contingency plan(s); explicit and specific statement documenting the intent to ensure continuity of the digital objects, and the steps taken and to be taken to ensure continuity; explicit agreements with successor organizations documenting the measures to be taken to ensure the complete and formal transfer of responsibility for the digital content and related assets, and granting the requisite rights necessary to ensure continuity of the content and repository services.

Discussion
A repository’s inability to preserve specific digital information threatens the long-term sustainability that information content. It is not sufficient for the repository to have an informal plan or policy regarding where its data goes should a failure occur. A formal plan with identified procedures needs to be in place.
Comment 5 Mark Conrad (mark.conrad.iso@gmail.com) 2020-05-15 17:57:34 UTC
(In reply to David Giaretta from comment #4)
> For specific wording, following John's suggestion to add a metric rather
> then change 3.1.2.1 add:
> 
> 4.3.5 The repository shall have an appropriate succession plan, contingency
> plans, and/or escrow arrangements in place in case the repository can no
> longer commit to long term preservation of specific digital information.
> 
> Supporting Text
> This is necessary in order to ensure that the objects it had originally
> committed to preserve will not be lost.
> 
> Examples of Ways the Repository Can Demonstrate It Is Meeting This
> Requirement
> Written and credible succession and contingency plan(s); explicit and
> specific statement documenting the intent to ensure continuity of the
> digital objects, and the steps taken and to be taken to ensure continuity;
> explicit agreements with successor organizations documenting the measures to
> be taken to ensure the complete and formal transfer of responsibility for
> the digital content and related assets, and granting the requisite rights
> necessary to ensure continuity of the content and repository services.
> 
> Discussion
> A repository’s inability to preserve specific digital information threatens
> the long-term sustainability that information content. It is not sufficient
> for the repository to have an informal plan or policy regarding where its
> data goes should a failure occur. A formal plan with identified procedures
> needs to be in place.

It is not practical to have succession plans, contingency plans, etc in place for each information object prior to the decision that the repository can no longer commit to long term preservation of that object. For one thing, the successor repository may not be able to commit to the long term preservation for similar reasons to those encountered by the original repository. 

It might be practical to negotiate with a successor repository when the original repository's technology watch indicates that a change is needed that is beyond the original repository's capabilities. The original repository should be able to perform bit preservation long enough to negotiate a new home for the data. 

The repository should also provide information to its Designated Communit(ies) when it is contemplating and after it has executed such a handoff.
Comment 6 David Giaretta (david@giaretta.org) 2020-05-23 19:39:12 UTC
(In reply to Mark Conrad from comment #5)
> (In reply to David Giaretta from comment #4)
> > For specific wording, following John's suggestion to add a metric rather
> > then change 3.1.2.1 add:
> > 
> > 4.3.5 The repository shall have an appropriate succession plan, contingency
> > plans, and/or escrow arrangements in place in case the repository can no
> > longer commit to long term preservation of specific digital information.
> > 
> > Supporting Text
> > This is necessary in order to ensure that the objects it had originally
> > committed to preserve will not be lost.
> > 
> > Examples of Ways the Repository Can Demonstrate It Is Meeting This
> > Requirement
> > Written and credible succession and contingency plan(s); explicit and
> > specific statement documenting the intent to ensure continuity of the
> > digital objects, and the steps taken and to be taken to ensure continuity;
> > explicit agreements with successor organizations documenting the measures
> to
> > be taken to ensure the complete and formal transfer of responsibility for
> > the digital content and related assets, and granting the requisite rights
> > necessary to ensure continuity of the content and repository services.
> > 
> > Discussion
> > A repository’s inability to preserve specific digital information threatens
> > the long-term sustainability that information content. It is not sufficient
> > for the repository to have an informal plan or policy regarding where its
> > data goes should a failure occur. A formal plan with identified procedures
> > needs to be in place.
> 
> It is not practical to have succession plans, contingency plans, etc in
> place for each information object prior to the decision that the repository
> can no longer commit to long term preservation of that object. For one
> thing, the successor repository may not be able to commit to the long term
> preservation for similar reasons to those encountered by the original
> repository. 
> 
> It might be practical to negotiate with a successor repository when the
> original repository's technology watch indicates that a change is needed
> that is beyond the original repository's capabilities. The original
> repository should be able to perform bit preservation long enough to
> negotiate a new home for the data. 
> 
> The repository should also provide information to its Designated
> Communit(ies) when it is contemplating and after it has executed such a
> handoff.

We could append to the Discussion:

It may not be practical to have firm, complete, succession plans in place for each information object or collection of information objects prior to the decision that the repository can no longer commit to long term preservation of that object. For example, potential successor repositories may not be able to commit to the long term preservation far in advance of the handover. It may be that interim escrow arrangements may need to be put in place.

It might, for example, be practical to negotiate with a successor repository when the original repository's technology watch indicates that a change is needed that is beyond the original repository's capabilities. The original repository should be able to perform at least bit preservation long enough to negotiate a new home for the data. 

The repository should also provide information to its Designated Communit(ies) when it is contemplating and after it has executed such a handoff.
Comment 7 David Giaretta (david@giaretta.org) 2020-05-26 11:01:12 UTC
*** Bug 119 has been marked as a duplicate of this bug. ***
Comment 8 Mark Conrad (mark.conrad.iso@gmail.com) 2020-06-01 18:22:53 UTC
The suggested text for the metric says that the repository SHALL have these measures in place. The suggested text to be added to the Discussion indicates that it may not be practical to do so. The two statements appear to be contradictory to me. 

If I have a repository with millions of digital objects in many different file formats covering many different domains, there is no way that I can negotiate in advance any kind of handover for each digital object. There would be no time to run the repository!

Like the suggested solution for SC 245, I think the best a repository could do in advance would be to have copies of all AIPs and Access Aids in a system independent format (i.e., not have to rely on export functions of existing systems) so that the information could be transferred to a successor repository on very short notice.
Comment 9 John Garrett (garrett@his.com) 2020-06-02 05:48:42 UTC
I'm not sure that I think it is important for a repository to have contingency plans for sending data somewhere else for all their data holdings.

For many repositories they probably expect to keep all (or at least most) of their holdings until the repository goes away.

Other repositories may expect that specific holdings are destroyed after a certain holding period.
Comment 10 David Giaretta (david@giaretta.org) 2020-06-02 09:24:50 UTC
(In reply to Mark Conrad from comment #8)
> The suggested text for the metric says that the repository SHALL have these
> measures in place. The suggested text to be added to the Discussion
> indicates that it may not be practical to do so. The two statements appear
> to be contradictory to me. 
> 
> If I have a repository with millions of digital objects in many different
> file formats covering many different domains, there is no way that I can
> negotiate in advance any kind of handover for each digital object. There
> would be no time to run the repository!
> 
> Like the suggested solution for SC 245, I think the best a repository could
> do in advance would be to have copies of all AIPs and Access Aids in a
> system independent format (i.e., not have to rely on export functions of
> existing systems) so that the information could be transferred to a
> successor repository on very short notice.

The test says:
"appropriate succession plan, contingency plans, and/or escrow arrangements in place"
i.e. it does not require a succession plan (which the Discussion refers to) - could be escrow or other contingency plans (unspecified), which really means that it should be pretty clear to the Designated Community, funders etc what will happen to the holdings i.e. it they are to be thrown in a rubbish skip then that should be made clear.
Comment 11 David Giaretta (david@giaretta.org) 2020-06-02 09:30:14 UTC
(In reply to John Garrett from comment #9)
> I'm not sure that I think it is important for a repository to have
> contingency plans for sending data somewhere else for all their data
> holdings.
> 
> For many repositories they probably expect to keep all (or at least most) of
> their holdings until the repository goes away.
> 
> Other repositories may expect that specific holdings are destroyed after a
> certain holding period.

See Comment #10 in reply to Mark.

If the repository expects (plans?) to destroy its holdings after a certain time, then OAIS allows this in the Mandatory Responsibilities as long as it follows procedures.

We could add to the Discussion:

"The contingency plans could include procedures to destroy certain of its holdings, following specified procedures"
Comment 12 David Giaretta (david@giaretta.org) 2020-09-28 13:59:19 UTC
Consolidating all the previous discussions I believe we have:

The following suggestion:
•	additional preservation techniques have been described explicitly in addition to Migration;

            ==> May be useful to add text to discussion of 4.3.1

OAIS talks about:
 - transformation
 - adding RepInfo
 - handover to another repository

ISO 16363 already has discussion of 
- transformation
- adding Representation.

Add new:

4.3.5	The repository shall have an appropriate succession plan, contingency plans, and/or escrow arrangements in place in case the repository can no longer commit to long term preservation of specific digital information.

Supporting Text
This is necessary in order to ensure that the objects it had originally committed to preserve will not be lost.

Examples of Ways the Repository Can Demonstrate It Is Meeting This Requirement
Written and credible succession and contingency plan(s); explicit and specific statement documenting the intent to ensure continuity of the digital objects, and the steps taken and to be taken to ensure continuity; explicit agreements with successor organizations documenting the measures to be taken to ensure the complete and formal transfer of responsibility for the digital content and related assets, and granting the requisite rights necessary to ensure continuity of the content and repository services.

Discussion
A repository’s inability to preserve specific digital information threatens the long-term sustainability that information content. It is not sufficient for the repository to have an informal plan or policy regarding where its data goes should a failure occur. A formal plan with identified procedures needs to be in place.
It may not be practical to have firm, complete, succession plans in place for each information object or collection of information objects prior to the decision that the repository can no longer commit to long term preservation of that object. For example, potential successor repositories may not be able to commit to the long term preservation far in advance of the handover. It may be that interim escrow arrangements may need to be put in place.
It might, for example, be practical to negotiate with a successor repository when the original repository's technology watch indicates that a change is needed that is beyond the original repository's capabilities. The original repository should be able to perform at least bit preservation long enough to negotiate a new home for the data. 
The repository should also provide information to its Designated Communit(ies) when it is contemplating and after it has executed such a handoff.
The contingency plans could include procedures to destroy certain of its holdings, following specified procedures
Comment 13 David Giaretta (david@giaretta.org) 2020-09-29 14:23:30 UTC
DAI20200929 Agreed to comment 12
Comment 14 A Maghsoudi (mapimp) 2023-04-07 17:20:39 UTC
Hello I’m I need help someone to make this work and need help someone please?
2.4.6 https://www.map.com is google maps and map map please please thank me for help thank you for coming back to help with you and help please help me help move please help please please thank love you for all family help.
Comment 15 David Giaretta (david@giaretta.org) 2023-04-11 14:31:54 UTC
Forgot to close