Oberon || Library || Module Index || Search Engine || Definition || Module


Ulm's Oberon Library:
Storage


NAME

Storage - storage allocating interface for the compiler

SYNOPSIS

TYPE Address = Types.UntracedAddress;
TYPE Size = Types.Size;
TYPE Status = POINTER TO StatusRec;
TYPE StatusRec =
   RECORD
      (Objects.ObjectRec)
      collection: INTEGER;
      intensity: INTEGER;
      allocated: Size;
      consumed: Size;
      chunksize: Size;
   END;
TYPE AllocProc = PROCEDURE(VAR ptr: Address; tag: Address; size: Size);
TYPE AllocStackProc = PROCEDURE(VAR ptr: Address; size: Size; mode: SHORTINT);
TYPE GarbageCollectionProc = PROCEDURE;
TYPE EnableProc = PROCEDURE;
TYPE DisableProc = PROCEDURE;
TYPE GetStatusProc = PROCEDURE (VAR status: Status);
TYPE IntensityProc = PROCEDURE (offset: INTEGER) : INTEGER;
TYPE ChunkSizeProc = PROCEDURE (size: Size) : Size;
TYPE FirstStackProc = PROCEDURE;
TYPE NextStackProc = PROCEDURE (VAR cr: Coroutines.Coroutine) : BOOLEAN;
TYPE Interface =
   RECORD
      alloc: AllocProc;
      allocStack: AllocStackProc;
      garbageCollection: GarbageCollectionProc;
      enable: EnableProc;
      disable: DisableProc;
      intensity: IntensityProc;
      getStatus: GetStatusProc;
      chunkSize: ChunkSizeProc;
      firstStack: FirstStackProc;
      nextStack: NextStackProc;
   END;


VAR end: Address; left: Size;

TYPE Event = POINTER TO EventRec; TYPE EventRec = RECORD (Events.EventRec) errors: RelatedEvents.Object; END; VAR outOfMemory: Events.EventType;

PROCEDURE AllocateWithTag(VAR ptr: Address; tag: Address; size: Size); PROCEDURE AllocateStack(VAR ptr: Address; size: Size; mode: SHORTINT);

PROCEDURE GarbageCollection; PROCEDURE EnableCollection; PROCEDURE DisableCollection;

PROCEDURE Intensity(offset: INTEGER) : INTEGER; PROCEDURE GetStatus(VAR status: Status); PROCEDURE ChunkSize(size: Size) : Size;

PROCEDURE FirstStack; PROCEDURE NextStack(VAR cr: Coroutines.Coroutine) : BOOLEAN;

PROCEDURE Init(if: Interface);

DESCRIPTION

Storage provides a system-independent storage allocating interface for the compiler and should not be used otherwise except as noted below. There may be different implementations for this interface, e.g. with or without garbage collector.

Init is to be called during startup by the module which is responsible for storage allocation (see SysStorage). All interface procedures have to follow the semantics described below.

The variables end and left define an interval [end-left, end) which may be consumed by the inline code of NEW and SYSTEM.NEW without calling any of the allocation procedures. Write accesses of these variables must be atomic either by using appropriate compare-and-set instructions of the native architecture or by blocking all asynchronous interrupts (see SysSignalOperations).

The runtime start sets up an initial region which should be sufficient to satisfy the memory requests in the early initialization stage. The module which calls Init is free to modify end and left. For all other modules these variables are to be considered read-only. Stacks must not be allocated by use of this region, and consequently coroutines must not be created during the early startup phase, i.e. before the storage allocator is initialized.

AllocateWithTag is to be called by the inline code of NEW and SYSTEM.NEW if the interval marked by end and left is not sufficient. size specifies the total number of bytes needed to store the record or array and the tag field and, if necessary, the dummy tag. The type tag is specified by tag which may be zero in case of SYSTEM.NEW. In this case a dummy tag record will be put behind the allocated area. The returned ptr points behind the stored type tag.

AllocateStack allocates a redzone-protected stack area for a coroutine with the given growing mode. Fixed redzone-protected regions may be requested by giving mode 0, but AllocateStack does not expect that such a region will become a coroutine stack. A negative value of mode specifies a backward growing stack. A positive value is to be given for forward growing stacks. On return, ptr points to the begin of the stack. For backward growing stacks this is the end address of the allocated region. The calling procedure or inline code is responsible for adjusting ptr and storing a tag reference to Coroutines.tag.

A call of GarbageCollection requests an immediate run of a garbage collection. Before the garbage collector is invoked, the event Process.startOfGarbageCollection is raised, while Process.endOfGarbageCollection signals the end of a collection. Normally, there is no need to call GarbageCollection, because the storage management is responsible for activating the garbage collector from time to time, but every module is free to do this, too.

GarbageCollection will have no effect, if there was a call of DisableCollection earlier. A single call of DisableCollection guarantees that the garbage collector is switched off. More than one calls in a row make it necessary that EnableCollection must be called as many times as DisableCollection before to re-enable garbage collections. The status record component collection can be interpreted as the number of times EnableCollection must be called to enable a garbage collection. At the beginning (i.e. after the storage management is initialized), garbage collection is enabled and collection is set to 0.

Likewise garbage collections can be suppressed if one or more coroutines are in a non-tracable state (see SysInterrupts).

The status record component intensity is an indicator for the current garbage collection frequency. This indicator can be interpreted as a clue to the amount of storage which is allowed to be consumed between two garbage collections. By default, when the intensity indicator is set to 0, that limit is equal to the current size of new memory chunks. Intensity allows to change the frequency of garbage collections by adding offset to the current frequency indicator. If offset is positive, the current collection frequency is decreased, while a negative value increases it. Please note, that Intensity(-1) causes the current garbage collection frequency to be doubled and Intensity(1) halves it! Intensity always returns the new frequency indicator. The following example sets the current intensity indicator to default value 0:

VAR defaultFrequency: INTEGER;
(* ... *)
defaultFrequency := Storage.Intensity(-Storage.Intensity(0));

GetStatus returns a pointer to a record which contains information about the current storage status. A positive value of component collection signals, that garbage collection is currently disabled. Garbage collection is enabled when collection is 0. The current garbage collection intensity indicator is returned in intensity. The chunksize component contains the size of new memory regions which will be requested from the operating system when more memory is needed. The value returned in allocated represents the total number of bytes which are currently under control of the storage allocator, while consumed contains the number of bytes which are currently in use (consumed<=allocated).

ChunkSize allows to change the current chunk size. The chunk size is the size of new memory chunks which the storage allocator requests from the system. There may be several system-dependent requirements which the chunk size has to meet, such as concerning address space utilization or the system's page size: ChunkSize rounds size to the next higher possible value, if the given parameter can not be used as new chunk size (generally, it should be a good idea when size is a power of 2 and greater than the page size). As result, the computed value is returned. In case of size = 0, ChunkSize only returns the current chunk size and no change is made.
Especially, when allocating objects with sizes greater than the page size, changing the chunk size can reduce the amount of memory needed from the operating system and increase performance. Using ChunkSize normally effects the current collection frequency, too, because it is not only dependent on the intensity indicator but also on the chunk size (Intensity(0) is performed).

FirstStack and NextStack allow to examine all coroutines. This can be useful for debugging purposes (see SysStacks and SysStackTraces).

DIAGNOSTICS

AllocateWithTag and AllocateStack always return valid pointers which need not to be checked against NIL. The storage allocator is responsible to raise the event outOfMemory short before it runs out of memory. The errors component of Storage.Event contains, if not equal to NIL, a queue of error events which were responsible for the ``out of memory'' event. The remaining memory should be sufficient to allow some event handling and a controlled termination. The storage allocator is expected to terminate the process immediately if, after raising outOfMemory, the last memory resources get consumed.

Calls to all interface procedures before Init lead to immediate program termination.

SEE ALSO

Coroutines
exports Coroutines.tag which marks coroutines.
Process
exports storage message events
SysStorage
storage allocator with copying garbage collection.
UntaggedStorage
alternative interface for storage unaffected by the garbage collection

BUGS

The default intensity of 0 is acceptable for very simple applications only. Everything else that is memory-intensive should raise this value significantly. Otherwise these applications would be more busy with garbage collection runs than their intended purpose. Raising the intensity reduces CPU time as the complexity of a garbage collection run depends only on the amount and size of living objects. However, memory consumption will raise significantly for higher intensity values. The optimal intensity value depends on the allocation behaviour of the application, the available virtual address space, and the available physical memory.

However, as large intensity values can lead to intensive swapping activities of the operating system, things can turn significantly worse by rendering the application or in some cases even the entire system unresponsive. Large intensity values can also lead to address space problems which lead to a panic exit of SysStorage.

Here are some rough estimates for the upper limits of intensity values in dependence of the physical memory that is available for this application:

memory   intensity
__________________
1 GB     9
800 MB   8
600 MB   7
400 MB   6

Larger values than 9 may cause address space problems if the application uses a considerable number of coroutines. This is an issue for networked applications that provide remote access to objects through RemoteObjects.

It is recommended for memory-intensive applications to include an option for the command line or configuration file that allows to set the intensity. This has been implemented, for example, by cdbd and obload.

AUTHOR

Hansjörg Nägele, University of Ulm

revisions are due to Andreas Borchert


Edited by: borchert, last change: 2005/08/23, revision: 1.6, converted to HTML: 2005/08/23

Oberon || Library || Module Index || Search Engine || Definition || Module