Mastering Delphi 7 [Electronic resources] نسخه متنی

Chapter 13, "Delphi's Database Architecture," I mentioned the possibility of writing a custom dataset class. Now it's time to look at an example. The reasons for writing a custom dataset relate to the fact that you won't need to deploy a database engine, but you'll still be able to take full advantage of Delphi's database architecture, including things like persistent database fields and data-aware controls.

Writing a custom dataset is one of the most complex tasks for a component developer, so this is one of the most advanced areas (as far as low-level coding practices, including tons of pointers) in the book. Moreover, Borland hasn't released any official documentation about writing custom datasets. If you are early in your experience with Delphi, you might want to skip the rest of this chapter and come back later.

TDataSet is an abstract class that declares several virtual abstract methods—23 in Delphi 5, now only a handful, as most have been replaced by empty virtual methods (which you still have to override). Every subclass of TDataSet must override all those methods.

Before discussing the development of a custom dataset, we need to explore a few technical elements of the TDataSet class—in particular, record buffering. The class maintains a list of buffers that store the values of different records. These buffers store the data, but they also usually store further information for the dataset to use when managing the records. These buffers don't have a predefined structure, and each custom dataset must allocate the buffers, fill them, and destroy them. The custom dataset must also copy the data from the record buffers to the various fields of the dataset, and vice versa. In other words, the custom dataset is entirely responsible for handling these buffers.

In addition to managing the data buffers, the component is responsible for navigating among the records, managing the bookmarks, defining the structure of the dataset, and creating the proper data fields. The TDataSet class is nothing more than a framework; you must fill it with the appropriate code. Fortunately, most of the code follows a standard structure, which the TDataSet-derived VCL classes use. Once you've grasped the key ideas, you'll be able to build multiple custom datasets borrowing quite a lot of code.

To simplify this type of reuse, I've collected the common features required by any custom dataset in a TMdCustomDataSet class. However, I'm not going to discuss the base class first and the specific implementation later, because it would be difficult to understand. Instead, I'll detail the code required by a dataset, presenting methods of the generic and specific classes at the same time, according to a logical flow.

The Definition of the Classes

The starting point, as usual, is the declaration of the two classes discussed in this section: the generic custom dataset I've written and a specific component storing data in a file stream. The declaration of these classes is available in Listing 17.2. In addition to virtual methods, the classes contain a series of protected fields used to manage the buffers, track the current position and record count, and handle many other features. You'll also notice another record declaration at the beginning: a structure used to store the extra data for every data record you place in a buffer. The dataset places this information in each record buffer, following the data.

Listing 17.2: The Declaration of TMdCustomDataSet and TMdDataSetStream

// in the unit MdDsCustom
type
EMdDataSetError = class (Exception);
TMdRecInfo = record
Bookmark: Longint;
BookmarkFlag: TBookmarkFlag;
end;
PMdRecInfo = ^TMdRecInfo;
TMdCustomDataSet = class(TDataSet)
protected
    // status
FIsTableOpen: Boolean;
    // record data
FRecordSize,        // the size of the actual data
FRecordBufferSize,  // data + housekeeping (TRecInfo)
FCurrentRecord,     // current record (0 to FRecordCount - 1)
BofCrack,           // before the first record (crack)
EofCrack: Integer;  // after the last record (crack)
    // create, close, and so on
procedure InternalOpen; override;
procedure InternalClose; override;
function IsCursorOpen: Boolean; override;
    // custom functions
function InternalRecordCount: Integer; virtual; abstract;
procedure InternalPreOpen; virtual;
procedure InternalAfterOpen; virtual;
procedure InternalLoadCurrentRecord(Buffer: PChar); virtual; abstract;
    // memory management
function AllocRecordBuffer: PChar; override;
procedure InternalInitRecord(Buffer: PChar); override;
procedure FreeRecordBuffer(var Buffer: PChar); override;
function GetRecordSize: Word; override;
    // movement and optional navigation (used by grids)
function GetRecord(Buffer: PChar; GetMode: TGetMode; DoCheck: Boolean):
TGetResult; override;
procedure InternalFirst; override;
procedure InternalLast; override;
function GetRecNo: Longint; override;
function GetRecordCount: Longint; override;
procedure SetRecNo(Value: Integer); override;
    // bookmarks
procedure InternalGotoBookmark(Bookmark: Pointer); override;
procedure InternalSetToRecord(Buffer: PChar); override;
procedure SetBookmarkData(Buffer: PChar; Data: Pointer); override;
procedure GetBookmarkData(Buffer: PChar; Data: Pointer); override;
procedure SetBookmarkFlag(Buffer: PChar;Value:TBookmarkFlag);override;
function GetBookmarkFlag(Buffer: PChar): TBookmarkFlag; override;
    // editing (dummy versions)
procedure InternalDelete; override;
procedure InternalAddRecord(Buffer: Pointer; Append: Boolean); override;
procedure InternalPost; override;
procedure InternalInsert; override;
    // other
procedure InternalHandleException; override;
published
    // redeclared dataset properties
property Active;
property BeforeOpen;
property AfterOpen;
property BeforeClose;
property AfterClose;
property BeforeInsert;
property AfterInsert;
property BeforeEdit;
property AfterEdit;
property BeforePost;
property AfterPost;
property BeforeCancel;
property AfterCancel;
property BeforeDelete;
property AfterDelete;
property BeforeScroll;
property AfterScroll;
property OnCalcFields;
property OnDeleteError;
property OnEditError;
property OnFilterRecord;
property OnNewRecord;
property OnPostError;
end;
// in the unit MdDsStream
type
TMdDataFileHeader = record
VersionNumber: Integer;
RecordSize: Integer;
RecordCount: Integer;
end;
TMdDataSetStream = class(TMdCustomDataSet)
private
procedure SetTableName(const Value: string);
protected
FDataFileHeader: TMdDataFileHeader;
FDataFileHeaderSize,    // optional file header size
FRecordCount: Integer;  // current number of records
FStream: TStream;       // the physical table
FTableName: string;     // table path and file name
FFieldOffset: TList;    // field offsets in the buffer
protected
    // open and close
procedure InternalPreOpen; override;
procedure InternalAfterOpen; override;
procedure InternalClose; override;
procedure InternalInitFieldDefs; override;
    // edit support
procedure InternalAddRecord(Buffer: Pointer; Append: Boolean); override;
procedure InternalPost; override;
procedure InternalInsert; override;
    // fields
procedure SetFieldData(Field: TField; Buffer: Pointer); override;
    // custom dataset virutal methods
function InternalRecordCount: Integer; override;
procedure InternalLoadCurrentRecord(Buffer: PChar); override;
public
procedure CreateTable;
function GetFieldData(Field:TField; Buffer:Pointer):Boolean;override;
published
property TableName: string read FTableName write SetTableName;
end;

When I divided the methods into sections (as you can see by looking at the source code files), I marked each one with a roman number. You'll see those numbers in a comment describing the method, so that while browsing this long listing you'll immediately know which of the four sections you are in.

Section I: Initialization, Opening, and Closing

The first methods I'll examine are responsible for initializing the dataset and for opening and closing the file stream used to store the data. In addition to initializing the component's internal data, these methods are responsible for initializing and connecting the proper TFields objects to the dataset component. To make this work, all you need to do is to initialize the FieldsDef property with the definitions of the fields for your dataset, and then call a few standard methods to generate and bind the TField objects. This is the general InternalOpen method:

procedure TMdCustomDataSet.InternalOpen;
begin
InternalPreOpen; // custom method for subclasses
  // initialize the field definitions
InternalInitFieldDefs;
  // if there are no persistent field objects, 
create the fields dynamically
  if DefaultFields then
CreateFields;
  // connect the TField objects with the actual fields
BindFields (True);
InternalAfterOpen; // custom method for subclasses
  // sets cracks and record position and size
BofCrack := -1;
EofCrack := InternalRecordCount;
FCurrentRecord := BofCrack;
FRecordBufferSize := FRecordSize + sizeof (TMdRecInfo);
BookmarkSize := sizeOf (Integer);
  // everything OK: table is now open
FIsTableOpen := True;
end;

You'll notice that the method sets most of the local fields of the class, and also the BookmarkSize field of the base TDataSet class. Within this method, I call two custom methods I introduced in my custom dataset hierarchy: InternalPreOpen and InternalAfterOpen. The first, InternalPreOpen, is used for operations required at the very beginning, such as checking whether the dataset can be opened and reading the header information from the file. The code checks an internal version number for consistency with the value saved when the table is first created, as you'll see later. By raising an exception in this method, you can eventually stop the open operation.

Here is the code for the two methods in the derived stream-based dataset:

const
HeaderVersion = 10;
procedure TMdDataSetStream.InternalPreOpen;
begin
  // the size of the header
FDataFileHeaderSize := sizeOf (TMdDataFileHeader);
  // check if the file exists
if not FileExists (FTableName) then
raise EMdDataSetError.Create ('Open: Table file not found');
  // create a stream for the file
FStream := TFileStream.Create (FTableName, fmOpenReadWrite);
// initialize local data (loading the header)
FStream.ReadBuffer (FDataFileHeader, FDataFileHeaderSize);
if FDataFileHeader.VersionNumber <> HeaderVersion then
raise EMdDataSetError.Create ('Illegal File Version');
  // let's read this, double check later
FRecordCount := FDataFileHeader.RecordCount;
end;
procedure TMdDataSetStream.InternalAfterOpen;
begin
  // check the record size
if FDataFileHeader.RecordSize <> FRecordSize then
raise EMdDataSetError.Create ('File record size mismatch');
  // check the number of records against the file size
if (FDataFileHeaderSize + FRecordCount * FRecordSize) <> FStream.Size 
then
raise EMdDataSetError.Create ('InternalOpen: Invalid Record Size');
end;

The second method, InternalAfterOpen, is used for operations required after the field definitions have been set and is followed by code that compares the record size read from the file against the value computed in the InternalInitFieldDefs method. The code also checks that the number of records read from the header is compatible with the size of the file. This test can fail if the dataset wasn't closed properly: You might want to modify this code to let the dataset refresh the record size in the header anyway.

The InternalOpen method of the custom dataset class is specifically responsible for calling InternalInitFieldDefs, which determines the field definitions (at either design time or run time). For this example, I decided to base the field definitions on an external file—an INI file that provides a section for every field. Each section contains the name and data type of the field, as well as its size if it is string data. Listing 17.3 shows the Contrib.INI file used in the component's demo application.

Listing 17.3: The Contrib.INI File for the Demo Application

[Fields]
Number = 6
[Field1]
Type = ftString
Name = Name
Size = 30
[Field2]
Type = ftInteger
Name = Level
[Field3]
Type = ftDate
Name = BirthDate
[Field4]
Type = ftCurrency
Name = Stipend
[Field5]
Type = ftString
Name = Email
Size = 50
[Field6]
Type = ftBoolean
Name = Editor

This file, or a similar one, must use the same name as the table file and must be in the same directory. The InternalInitFieldDefs method (shown in Listing 17.4) will read it, using the values it finds to set up the field definitions and determine the size of each record. The method also initializes an internal TList object that stores the offset of every field inside the record. You use this TList to access fields' data within the record buffer, as you can see in the code listing.

Listing 17.4: The InternalInitFieldDefs Method of the Stream-Based Dataset

procedure TMdDataSetStream.InternalInitFieldDefs;
var
IniFileName, FieldName: string;
IniFile: TIniFile;
nFields, I, TmpFieldOffset, nSize: Integer;
FieldType: TFieldType;
begin
FFieldOffset := TList.Create;
FieldDefs.Clear;
TmpFieldOffset := 0;
IniFilename := ChangeFileExt(FTableName, '.ini');
Inifile := TIniFile.Create (IniFilename);
  // protect INI file
try
nFields := IniFile.ReadInteger (' Fields', 'Number', 0);
if nFields = 0 then
raise EDataSetOneError.Create (' InitFieldsDefs: 0 fields?');
for I := 1 to nFields do
begin
      // create the field
FieldType := TFieldType (GetEnumValue (TypeInfo (TFieldType),
IniFile.ReadString ('Field' + IntToStr (I), 'Type', '')));
FieldName := IniFile.ReadString ('Field' + IntToStr (I), 'Name', '');
if FieldName = ''  then
raise EDataSetOneError.Create (
'InitFieldsDefs: No name for field ' + IntToStr (I));
nSize := IniFile.ReadInteger ('Field' + IntToStr (I), 'Size', 0);
FieldDefs.Add (FieldName, FieldType, nSize, False);
      // save offset and compute size
FFieldOffset.Add (Pointer (TmpFieldOffset));
case FieldType of
ftString:                         Inc (TmpFieldOffset, nSize + 1);
ftBoolean, ftSmallInt, ftWord:    Inc (TmpFieldOffset, 2);
ftInteger, ftDate, ftTime:        Inc (TmpFieldOffset, 4);
ftFloat, ftCurrency, ftDateTime:  Inc (TmpFieldOffset, 8);
else
raise EDataSetOneError.Create (
'InitFieldsDefs: Unsupported field type');
end;
end; // for
finally
IniFile.Free;
end;
FRecordSize := TmpFieldOffset;
end;

Closing the table is a matter of disconnecting the fields (using some standard calls). Each class must dispose of the data it allocated and update the file header, the first time records are added and each time the record count has changed:

procedure TMdCustomDataSet.InternalClose;
begin
// disconnect field objects
BindFields (False);
  // destroy field object (if not persistent)
if DefaultFields then
DestroyFields;
  // close the file
FIsTableOpen := False;
end;
procedure TMdDataSetStream.InternalClose;
begin
  // if required, save updated header
if (FDataFileHeader.RecordCount <> FRecordCount) or
(FDataFileHeader.RecordSize = 0) then
begin
FDataFileHeader.RecordSize := FRecordSize;
FDataFileHeader.RecordCount := FRecordCount;
if Assigned (FStream) then
begin
FStream.Seek (0, soFromBeginning);
FStream.WriteBuffer (FDataFileHeader, FDataFileHeaderSize);
end;
end;
  // free the internal list field offsets and the stream
FFieldOffset.Free;
FStream.Free;
  inherited InternalClose;
end;

Another related function is used to test whether the dataset is open, something you can solve using the corresponding local field:

function TMdCustomDataSet.IsCursorOpen: Boolean;
begin
Result := FIsTableOpen;
end;

These are the opening and closing methods you need to implement in any custom dataset. However, most of the time, you'll also add a method to create the table. In this example, the CreateTable method creates an empty file and inserts information in the header: a fixed version number, a dummy record size (you don't know the size until you initialize the fields), and the record count (which is zero to start):

procedure TMdDataSetStream.CreateTable;
begin
CheckInactive;
InternalInitFieldDefs;
  // create the new file
if FileExists (FTableName) then
raise EMdDataSetError.Create('File '+ FTableName +' already exists');
FStream := TFileStream.Create (FTableName, fmCreate 
or fmShareExclusive);
try
    // save the header
FDataFileHeader.VersionNumber := HeaderVersion;
FDataFileHeader.RecordSize := 0;    // used later
FDataFileHeader.RecordCount := 0;   // empty
FStream.WriteBuffer (FDataFileHeader, FDataFileHeaderSize);
finally
    // close the file
FStream.Free;
  end;
end;

Section II: Movement and Bookmark Management

As mentioned earlier, every dataset must implement bookmark management, which is necessary for navigating through the dataset. Logically, a bookmark is a reference to a specific dataset record, something that uniquely identifies the record so a dataset can access it and compare it to other records. Technically, bookmarks are pointers. You can implement them as pointers to specific data structures that store record information, or you can implement them as record numbers. For simplicity, I'll use the latter approach.

Given a bookmark, you should be able to find the corresponding record; but given a record buffer, you should also be able to retrieve the corresponding bookmark. This is the reason for appending the TMdRecInfo structure to the record data in each record buffer. This data structure stores the bookmark for the record in the buffer, as well as some bookmark flags defined as follows:

type
TBookmarkFlag = (bfCurrent, bfBOF, bfEOF, bfInserted);

The system will request that you store these flags in each record buffer and will later ask you to retrieve the flags for a given record buffer.

To summarize, the structure of a record buffer stores the record data, the bookmark, and the bookmark flags, as you can see in Figure 17.5.

Figure 17.5: The structure of each buffer of the custom dataset, along with the various local fields referring to its subportions

To access the bookmark and flags, you can use as an offset the size of the data, casting the value to the PMdRecInfo pointer type, and then access the proper field of the TMdRecInfo structure via the pointer. The two methods used to set and get the bookmark flags demonstrate this technique:

procedure TMdCustomDataSet.SetBookmarkFlag (Buffer: PChar;
Value: TBookmarkFlag);
begin
PMdRecInfo(Buffer + FRecordSize).BookmarkFlag := Value;
end;
function TMdCustomDataSet.
GetBookmarkFlag (Buffer: PChar): TBookmarkFlag;
begin
Result := PMdRecInfo(Buffer + FRecordSize).BookmarkFlag;
end;

The methods you use to set and get a record's current bookmark are similar to the previous two, but they add complexity because you receive a pointer to the bookmark in the Data parameter. Casting the value referenced by this pointer to an integer, you obtain the bookmark value:

procedure TMdCustomDataSet.GetBookmarkData (Buffer: 
PChar; Data: Pointer);
begin
Integer(Data^) := PMdRecInfo(Buffer + FRecordSize).Bookmark;
end;
procedure TMdCustomDataSet.SetBookmarkData (Buffer: 
PChar; Data: Pointer);
begin
PMdRecInfo(Buffer + FRecordSize).Bookmark := Integer(Data^);
end;

The key bookmark management method is InternalGotoBookmark, which your dataset uses to make a given record the current one. This isn't the standard navigation technique—it's much more common to move to the next or previous record (something you can accomplish using the GetRecord method presented in the next section), or to move to the first or last record (something you'll accomplish using the InternalFirst and InternalLast methods described shortly).

Oddly enough, the InternalGotoBookmark method doesn't expect a bookmark parameter, but a pointer to a bookmark, so you must dereference it to determine the bookmark value. You use the following method, InternalSetToRecord, to jump to a given bookmark, but it must extract the bookmark from the record buffer passed as a parameter. Then, InternalSetToRecord calls InternalGotoBookmark. Here are the two methods:

procedure TMdCustomDataSet.InternalGotoBookmark (Bookmark: Pointer);
var
ReqBookmark: Integer;
begin
ReqBookmark := Integer (Bookmark^);
if (ReqBookmark >= 0) and (ReqBookmark < InternalRecordCount) then
FCurrentRecord := ReqBookmark
  else
raise EMdDataSetError.Create ('Bookmark ' +
IntToStr (ReqBookmark) + ' not found');
end;
procedure TMdCustomDataSet.InternalSetToRecord (Buffer: PChar);
var
ReqBookmark: Integer;
begin
ReqBookmark := PMdRecInfo(Buffer + FRecordSize).Bookmark;
InternalGotoBookmark (@ReqBookmark);
end;

In addition to the bookmark management methods just described, you use several other navigation methods to move to specific positions within the dataset, such as the first or last record. These two methods don't really move the current record pointer to the first or last record, but move it to one of two special locations before the first record and after the last one. These are not actual records: Borland calls them cracks. The beginning-of-file crack, or BofCrack, has the value –1 (set in the InternalOpen method), because the position of the first record is zero. The end-of-file crack, or EofCrack, has the value of the number of records, because the last record has the position FRecordCount - 1. I used two local fields, called EofCrack and BofCrack, to make this code easier to read:

procedure TMdCustomDataSet.InternalFirst;
begin
FCurrentRecord := BofCrack;
end;
procedure TMdCustomDataSet.InternalLast;
begin
EofCrack := InternalRecordCount;
FCurrentRecord := EofCrack;
end;

The InternalRecordCount method is a virtual method introduced in my TMdCustomDataSet class, because different datasets can either have a local field for this value (as in case of the stream-based dataset, which has an FRecordCount field) or compute it on the fly.

Another group of optional methods is used to get the current record number (used by the DBGrid component to show a proportional vertical scroll bar), set the current record number, or determine the number of records. These methods are easy to understand, if you recall that the range of the internal FCurrentRecord field is from 0 to the number of records minus 1. In contrast, the record number reported to the system ranges from 1 to the number of records:

function TMdCustomDataSet.GetRecordCount: Longint;
begin
CheckActive;
Result := InternalRecordCount;
end;
function TMdCustomDataSet.GetRecNo: Longint;
begin
UpdateCursorPos;
if FCurrentRecord < 0 then
Result := 1
else
Result := FCurrentRecord + 1;
end;
procedure TMdCustomDataSet.SetRecNo(Value: Integer);
begin
CheckBrowseMode;
if (Value > 1) and (Value <= FRecordCount) then
begin
FCurrentRecord := Value - 1;
Resync([]);
end;
end;

Notice that the generic custom dataset class implements all the methods of this section. The derived stream-based dataset doesn't need to modify any of them.

Section III: Record Buffers and Field Management

Now that we've covered all the support methods, let's examine the core of a custom dataset. In addition to opening and creating records and moving around between them, the component needs to move the data from the stream (the persistent file) to the record buffers, and from the record buffers to the TField objects that are connected to the data-aware controls. The management of record buffers is complex, because each dataset also needs to allocate, empty, and free the memory it requires:

function TMdCustomDataSet.AllocRecordBuffer: PChar;
begin
GetMem (Result, FRecordBufferSize);
end;
procedure TMdCustomDataSet.FreeRecordBuffer (var Buffer: PChar);
begin
FreeMem (Buffer);
end;

You allocate memory this way because a dataset generally adds more information to the record buffer, so the system has no way of knowing how much memory to allocate. Notice that in the AllocRecordBuffer method, the component allocates the memory for the record buffer, including both the database data and the record information. In the InternalOpen method, I wrote the following:

FRecordBufferSize := InternalRecordSize + sizeof (TMdRecInfo);

The component also needs to implement a function to reset the buffer (InternalInitRecord), usually filling it with numeric zeros or spaces.

Oddly enough, you must also implement a method that returns the size of each record, but only the data portion—not the entire record buffer. This method is necessary for implementing the read-only RecordSize property, which is used only in a couple of peculiar cases in the entire VCL source code. In the generic custom dataset, the GetRecordSize method returns the value of the FRecordSize field.

Now we've reached the core of the custom dataset component. The methods in this group are GetRecord, which reads data from the file; InternalPost and InternalAddRecord, which update or add new data to the file; and InternalDelete, which removes data and is not implemented in the sample dataset.

The most complex method of this group is GetRecord, which serves multiple purposes. The system uses this method to retrieve the data for the current record, fill a buffer passed as a parameter, and retrieve the data of the next or previous records. The GetMode parameter determines its action:

type
TGetMode = (gmCurrent, gmNext, gmPrior);

Of course, a previous or next record might not exist. Even the current record might not exist—for example, when the table is empty (or in case of an internal error). In these cases you don't retrieve the data but return an error code. Therefore, this method's result can be one of the following values:

type
TGetResult = (grOK, grBOF, grEOF, grError);

Checking to see if the requested record exists is slightly different than you might expect. You don't have to determine if the current record is in the proper range, only if the requested record is. For example, in the gmCurrent branch of the case statement, you use the standard expression CurrentRecord>= InternalRecourdCount. To fully understand the various cases, you might want to read the code a couple of times.

It took me some trial and error (and system crashes caused by recursive calls) to get the code straight when I wrote my first custom dataset a few years back. To test it, consider that if you use a DBGrid, the system will perform a series of GetRecord calls, until either the grid is full or GetRecord return grEOF. Here's the entire code for the GetRecord method:

// III: Retrieve data for current, previous, or next record
// (moving to it if necessary) and return the status
function TMdCustomDataSet.GetRecord(Buffer: PChar;
GetMode: TGetMode; DoCheck: Boolean): TGetResult;
begin
Result := grOK; // default
case GetMode of
gmNext: // move on
if FCurrentRecord < InternalRecordCount - 1 then
Inc (FCurrentRecord)
else
Result := grEOF; // end of file
gmPrior: // move back
if FCurrentRecord > 0 then
Dec (FCurrentRecord)
else
Result := grBOF; // begin of file
gmCurrent: // check if empty
if FCurrentRecord >= InternalRecordCount then
Result := grError;
end;
  // load the data
if Result = grOK then
InternalLoadCurrentRecord (Buffer)
else if (Result = grError) and DoCheck then
raise EMdDataSetError.Create ('GetRecord: Invalid record');
end;

If there's an error and the DoCheck parameter was True, GetRecord raises an exception. If everything goes fine during record selection, the component loads the data from the stream, moving to the position of the current record (given by the record size multiplied by the record number). In addition, you need to initialize the buffer with the proper bookmark flag and bookmark (or record number) value. This is accomplished by another virtual method I introduced, so that derived classes will only need to implement this portion of the code, while the complex GetRecord method remains unchanged:

procedure TMdDataSetStream.InternalLoadCurrentRecord (Buffer: PChar);
begin
FStream.Position := FDataFileHeaderSize + FRecordSize * FCurrentRecord;
FStream.ReadBuffer (Buffer^, FRecordSize);
with PMdRecInfo(Buffer + FRecordSize)^ do
  begin
BookmarkFlag := bfCurrent;
Bookmark := FCurrentRecord;
  end;
end;

You move data to the file in two different cases: when you modify the current record (that is, a post after an edit) or when you add a new record (a post after an insert or append). You use the InternalPost method in both cases, but you can check the dataset's State property to determine which type of post you're performing. In both cases, you don't receive a record buffer as a parameter; so, you must use the ActiveRecord property of TDataSet, which points to the buffer for the current record:

procedure TMdDataSetStream.InternalPost;
begin
CheckActive;
if State = dsEdit then
begin
// replace data with new data
FStream.Position := FDataFileHeaderSize + FRecordSize * FCurrentRecord;
FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
end
else
begin
    // always append
InternalLast;
FStream.Seek (0, soFromEnd);
FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
Inc (FRecordCount);
end;
end;

In addition, there's another related method: InternalAddRecord. This method is called by the AddRecord method, which in turn is called by InsertRecord and AppendRecord. These last two are public methods a user can call. This is an alternative to inserting or appending a new record to the dataset, editing the values of the various fields, and then posting the data, because the InsertRecord and AppendRecord calls receive the values of the fields as parameters. All you must do at that point is replicate the code used to add a new record in the InternalPost method:

procedure TMdDataSetOne.InternalAddRecord(Buffer:
 Pointer; Append: Boolean);
begin
// always append at the end
InternalLast;
FStream.Seek (0, soFromEnd);
FStream.WriteBuffer (ActiveBuffer^, FRecordSize);
Inc (FRecordCount);
end;

I should also have implemented a file operation that removes the current record. This operation is common, but it is complex. If you take a simple approach, such as creating an empty spot in the file, then you'll need to keep track of that spot and make the code for reading or writing a specific record work around that location. An alternate solution is to make a copy of the entire file without the given record and then replace the original file with the copy. Given these choices, I felt that for this example I could forgo supporting record deletion.

Section IV: From Buffers to Fields

In the last few methods, you've seen how datasets move data from the data file to the memory buffer. However, there's little Delphi can do with this record buffer, because it doesn't yet know how to interpret the data in the buffer. You need to provide two more methods: GetData, which copies the data from the record buffer to the field objects of the dataset, and SetData, which moves the data back from the fields to the record buffer. Delphi will automatically move the data from the field objects to the data-aware controls and back.

The code for these two methods isn't difficult, primarily because you saved the field offsets inside the record data in a TList object called FFieldOffset. By incrementing the pointer to the initial position in the record buffer of the current field's offset, you can get the specific data, which takes Field.DataSize bytes.

A confusing element of these two methods is that they both accept a Field parameter and a Buffer parameter. At first, you might think the buffer passed as a parameter is the record buffer. However, I found out that the Buffer is a pointer to the field object's raw data. If you use one of the field object's methods to move that data, it will call the dataset's GetData or SetData method, probably causing an infinite recursion. Instead, you should use the ActiveBuffer pointer to access the record buffer, use the proper offset to get to the data for the current field in the record buffer, and then use the provided Buffer to access the field data. The only difference between the two methods is the direction you move the data:

function TMdDataSetOne.GetFieldData (Field:
 TField; Buffer: Pointer): Boolean;
var
FieldOffset: Integer;
Ptr: PChar;
begin
Result := False;
if not IsEmpty and (Field.FieldNo > 0) then
begin
FieldOffset := Integer (FFieldOffset [Field.FieldNo - 1]);
Ptr := ActiveBuffer;
Inc (Ptr, FieldOffset);
if Assigned (Buffer) then
Move (Ptr^, Buffer^, Field.DataSize);
Result := True;
if (Field is TDateTimeField) and (Integer(Ptr^) = 0) then
Result := False;
end;
end;
procedure TMdDataSetOne.SetFieldData(Field: TField; Buffer: Pointer);
var
FieldOffset: Integer;
Ptr: PChar;
begin
if Field.FieldNo >= 0 then
begin
FieldOffset := Integer (FFieldOffset [Field.FieldNo - 1]);
Ptr := ActiveBuffer;
Inc (Ptr, FieldOffset);
if Assigned (Buffer) then
Move (Buffer^, Ptr^, Field.DataSize)
else
      raise Exception.Create (
'Very bad error in TMdDataSetStream.SetField data');
DataEvent (deFieldChange, Longint(Field));
end;
end;

The GetField method should return True or False to indicate whether the field contains data or is empty (a null field, to be more precise). However, unless you use a special marker for blank fields, it's difficult to determine this condition, because you're storing values of different data types. For example, a test such as Ptr^<>#0 makes sense only if you are using a string representation for all the fields. If you use this test, zero integer values and empty strings will show as null values (the data-aware controls will be empty), which may be what you want. The problem is that Boolean False values won't show up. Even worse, floating-point values with no decimals and few digits won't be displayed, because the exponent portion of their representation will be zero. However, to make this example work, I had to consider as empty each date/time field with an initial zero. Without this code, Delphi tries to convert the illegal internal zero date (internally, date fields don't use a TDateTime data type but a different representation), raising an exception. The code used to work with past versions of Delphi.

There's one final method, which doesn't fall into any category: InternalHandleException. Generally, this method silences the exception, because it is activated only at design time.

Testing the Stream-Based DataSet

After all this work, you're ready to test an application example of the custom dataset component, which is installed in the component's package for this chapter. The form displayed by the StreamDSDemo program is simple, as you can see in Figure 17.6. It has a panel with two buttons, a check box, and a navigator component, plus a DBGrid filling its client area.

Figure 17.6: The form of the StreamDSDemo example. The custom dataset has been activated, so you can already see the data at design time.

Figure 17.6 shows the example's form at design time, but I activated the custom dataset so that its data is visible. I already prepared the INI file with the table definition (the file listed earlier when discussing the dataset initialization), and I executed the program to add some data to the file.

You can also modify the form using Delphi's Fields editor and set the properties of the various field objects. Everything works as it does with one of the standard dataset controls. However, to make this work, you must enter the name of the custom dataset's file in the TableName property, using the complete path.

The example code is simple, especially compared to the custom dataset code. If the table doesn't exist yet, you can click the Create New Table button:

procedure TForm1.Button1Click(Sender: TObject);
begin
MdDataSetStream1.CreateTable;
MdDataSetStream1.Open;
CheckBox1.Checked := MdDataSetStream1.Active;
end;

You create the file first, opening and closing it within the CreateTable call, and then open the table. This is the same behavior as the TTable component (which accomplishes this step using the CreateTable method). To open or close the table, you can click the check box:

procedure TForm1.CheckBox1Click(Sender: TObject);
begin
MdDataSetStream1.Active := CheckBox1.Checked;
end;

Finally, I created a method that tests the custom dataset's bookmark management code (it works).