################################################################

$RCSfile: README.txt,v $

Author: Craeg Strong

$Date: 2003/04/21 01:52:57 $

################################################################


Contents

  1. Description

  2. Default Behavior

  3. Extensibility

  4. Creating Instances Programmatically

  5. Known Limitations

  6. Contributions

  7. Schema Migration

  8. Notes

  9. Unit Testing on Win32

Description

  This is the ExternalFile Zope Product.  It is similar to ExtFile
  (see "ExtFile":http://www.zope.org/Members/MacGregor/ExtFile ),
  except that it references the file in situ, rather than copying it
  to a file system repository within the context of the Zope
  installation.  It also supports in-browser editing for ASCII files.

  An ExternalFile is a reference to a file in the filesystem, like a
  symbolic link in a UNIX file system (or windows shortcut).  You can
  create multiple instances of ExternalFile pointing to the same
  physical file.

  The file referred to by an instance of ExternalFile must be
  accessible to the Zope server.  This usually means that it must
  reside in a local filesystem, or possibly one that is mounted on the
  Zope server machine through NFS, Samba, Novell, or some such
  protocol.  It also means that Zope must have permission to access
  it.  Since Zope is usually running as an administrative user or some
  special user, this can be a bit tricky.  For example, on UNIX
  systems Zope is usually running as "nobody."  One possible solution
  might be to place the Zope user in the same group as the developers.
  This way they can view and edit their own files online as long as
  they make them group writable.  Of course, if you start up your own
  private Zope server on your local system, you won't encounter any of
  these issues.

  Why refer to the files in place?  There are many possible reasons,
  one of which is to support the usage of a sandbox-oriented version
  control system.  By creating instances of ExternalFile pointing to
  the content files in the developer sandbox, one can continue to use
  the version control system in a very natural way.  

  There are two ways to create ExternalFiles from the Zope Management
  Interface (ZMI): one at a time or in a batch.  To use the batch
  option, simply select "External File Batch" from the standard Zope
  'Add' menu.  It provides a way to automatically add an instance of
  ExternalFile for every file within a directory.  Because
  ExternalFile instances are merely links or aliases, they can be
  deleted without affecting the underlying file.  Therefore, an easy
  way to create ExternalFile instances is to create them a batch at a
  time and then delete the unwanted instances (for example, those
  pointing to junk files, "tilde files", etc.).

  ExternalFile is intended to be subclassed.  For example, CVSFile is
  a Zope Product that inherits from ExternalFile and adds support for
  working with external files within a CVS sandbox.

  ExternalFile can behave like standard Zope Products.  The current
  list is:

  - DTMLDocument

  - DTMLMethod

  - Image

  - File

  - PageTemplate

  When an ExternalFile is set to 'behave_like' a DTMLDocument, for
  example, it should be able to be used in place of a DTMLDocument.
  The same goes for the others in the list above.  If you find that an
  ExternalFile with a certain 'behave_like' setting does *not* work
  the same way as the "real" Zope object, please report this as a bug.
  A small testcase demonstrating the problem would be *greatly*
  appreciated.  The one known exception is that ExternalFile does not
  currently support webDAV locking.

  ExternalFile now *requires* Page Templates.  For Zope releases 2.5
  and above, Page Templates are available by default.  For earlier
  Zope releases they must be added by separately installing the
  ZopePageTemplates package.

  ExternalFile has been tested in a production website setting, with
  four active content authors, 150 pages of content, and 100 hits per
  day.  It may therefore be considered as being stable and production
  ready.

  ExternalFile encapsulates a file in the filesystem for use as a Zope
  object.  The filesystem must be locally accessible to the Zope
  server.  The metadata associated with the ExternalFile is stored in
  the Zope object database, but the content itself is not.  When the
  content of an ExternalFile is accessed by Zope, it is automatically
  retrieved from the filesystem.  An ExternalFile always points to a
  file that exists locally on the server.  That file may be:

  - already existing on the server,

  - created by uploading a local file to the server and
    specifying a target filepath,

  - created by uploading a local file to the server and
    overwriting a file already existing on the server, or

  - created on the server, with initially empty content.

Default Behavior

  The behavior of an ExternalFile can be changed at any time, simply
  by modifying the 'behave_like' property in the Properties tab in the
  Zope Management Interface.  However, when an ExternalFile is first
  created, the default setting for 'behave_like' is chosen according
  to the mimetype of the target file.  For a precise description, see
  "Display Types", but the table below provides a rough idea::

    Content-Type    Default 'behave_like'

    ============    =====================

    HTML,DTML       DTMLDocument

    JPG,GIF,BMP     Image

    Other           File

Extensibility

  When a Zope server that has the ExternalFile product installed is
  first started, it loads in two ASCII files, located in the 'etc'
  subdirectory:
  '[ZOPE]/lib/python/Products/ExternalFile/etc/mime.types' and
  '[ZOPE]/lib/python/Products/ExternalFile/etc/display.types'.  Feel
  free to customize these files as you see fit for your installation.

    *CAVEAT: Understand, however, that *removing* entries from one of
    the files might produce unexpected results, for example, if you
    had an existing instance of ExternalFile using a mimetype listed
    in but then removed from 'mime.types'.  Don't do that.*

  Mime Types

    Each instance of ExternalFile can be assigned a mimetype from a list
    in the Properties tab of the Zope Management Interface.

    This list is created by adding the types listed in
    'mime.types' to those available by default in Python.  To view
    the current list of Python-default mimetypes, look
    "here":http://www.isi.edu/in-notes/iana/assignments/media-types/media-types
    *N.B. That is not actually the list, but should be roughly close.
    I have not been able to find the list online.  It is available on
    page 268 of the Python Essential Reference, Second Edition.*

    The format of the 'mime.types' file is the same as that used by
    the Apache web server (de facto standard).  Each line consists of
    a mimetype and zero, one, or more file extensions.  The file
    extensions are used to guess mimetypes from files.  For example,
    if you add a new Zope object called 'foo.jpg', the extension 'jpg'
    will be looked up in the map to retrieve the 'image/jpeg'
    mimetype.  Here is an example entry::

    text/css			css  # Cascading stylesheets

    Instances of ExternalFile (and therefore its subclasses, such as
    CVSFile) store their mimetype persistently as a property.  The
    mimetype may be changed through the Properties tab in the ZMI.

  Display Types

    DisplayType is an invented concept that is used *solely* to decide
    how to edit and view ExternalFiles in the ZMI.  There are three
    flavors of DisplayType::

       ascii   -> Editable Though The Web, viewable via HTML readonly text area

       image   -> Not editable TTW, viewable via HTML image tag

       binary  -> Not editable TTW, viewable via HTML hyperlink

    Generally, the DisplayType is derived by parsing the mimetype into
    'type/subtype'.  That is::

      image/jpeg       -> image

      text/plain       -> ascii

      (anything else)  -> binary

    In certain circumstances, the algorithm does not yield correct
    results.  For example, 'application/xml' should be 'ascii' rather
    than 'binary.'  'display.types' is used for those
    circumstances i.e. when the algorithm would yield incorrect
    results.  You may add to it or modify as needed.

    For example, you may find that your browser cannot display certain
    unusual image types correctly in &lt;img&gt; tags.  This could be
    corrected by an entry such as::

      image/x-cmu-raster	binary

    Instances of ExternalFile do *not* maintain their 'displaytype'
    persistently, rather they query ProductProperties at run-time for
    their 'displaytype' based on their 'mimetype'.  That means that
    changing the entries in 'display.types' will change the
    behavior of all instances of ExternalFile (and those of its
    subclasses, such as CVSFile).

Creating Instances Programmatically

  Normally instances of ExternalFile are created by hand in the Zope
  Management Interface.  There are a number of screens (a
  mini-workflow) that are traversed in order to ensure that the
  instance is created properly, to deal with all of the things that
  can go wrong (creating new files, overwriting existing files,
  permission issues, etc.) and to confirm exactly what was done.

  However, there are times when you don't want that.  You simply want
  to run a Python script and create an instance of ExternalFile.  Do
  not despair!  This is easy to do.  Check out the 'extras'
  subdirectory.  You will find the source code for an example:
  'uploadCreate'.  It is commented and should be easy to adapt to your
  uses.  Basically it calls the 'manage_addExternalFile()' method in
  'ExternalFile.py' directly, bypassing the GUI creation workflow code
  (which has all been moved into a separate module:
  'CreationDialog.py').

Known Limitations

  ExternalFiles do not support webDAV the way real DTMLDocuments do.
  However, they do support most other things such as through the web
  (TTW) editing, file upload, and FTP.  Frankly, we are not planning
  to fix this shortcoming until the whole thing is revisited in the
  context of Zope3.  In fact, rumor has it that the Zope3 object
  database will feature a file system synchronization technology that
  may even obviate ExternalFile!

  ExternalFiles do not handle security exactly the same way
  DTMLDocuments do.  This means that there are certain common-usage
  scenarios where ExternalFiles can NOT be used in place of
  DTMLDocuments.  We have not been able to nail down a simple
  reproducible test case for this as yet.  If you have one, please
  email it to us so we can fix this annoying bug.

Contributions

  We hope others find this code useful.  If you have extended or
  improved this product, please feel free to submit your changes to 
  "us":mailto:support@arielpartners.com.
  
  If there is enough interest, we would certainly consider setting up
  an open-source project on "SourceForge":http://www.sourceforge.net.

Schema Migration

  From time to time, you may find yourself with a new version of this
  product, either because we have released a "new improved" version or
  from some changes you may have made on your own.  How do you deal
  with all of the existing instances in your ZODB that were created
  with the old definition?  Here is our preferred technique:

  1. Find the repair() method (we usually put it at the bottom of the
     respective python source file)

  2. Change the repair() method so that it updates the object from the
     old version to the new version

  3. Create a python script in the root folder that recursively calls
     repair() on each of the object with a given metatype

  4. Execute the python script (the easy way is via the "test" tab)

  Here is our python script.  We call it "repairAll" and put it in the
  ZODB root folder::

    objects = context.ZopeFind(context, obj_metatypes=[metaType], search_sub=1)
    map(lambda x: x[1].repair(), objects)
    return map(lambda x: x[0], objects)

  In general, the repair() method is coded to convert an instance from
  the previous version to the current version.  To convert from an
  earlier version (that is, to skip a version) you may have to code it
  yourself.  It is actually quite easy to do, but don't hesitate to
  email if you need pointers or advice.

Notes

  This document is written in structured text.  For a quickie intro to
  structured text, look
  "here":http://www.zope.org/Documentation/Articles/STX.

Unit Testing on Win32

  If you don't plan to run the automated unit test suite, this
  section is irrelevant.

  For some reason, the automated unit tests don't run properly for me
  on win32 using the FAT file system under Zopes earlier than 2.5.1.
  I found the following workaround.  In the file name::

    [ZOPE]\lib\python\Testing\custom_zodb.py

  Where ZOPE stands for the directory in which you installed Zope,
  Change line number seven::

    Storage = DemoStorage(base=FileStorage(dfi,read_only=1), quota=(1<<20))

  to instead read::

    Storage = DemoStorage(base=FileStorage(dfi, read_only=0), quota=(1<<20))

  My guess is that FAT does not support the read_only attribute as
  required.
