Persisting data in a JDataStore

JDataStore is a feature of JBuilder Professional and Enterprise, and the Inprise Application Server.

The data-aware dbSwing and JBCL controls bind to DataExpress datasets. The DataExpress paradigm cleanly separates data sources from datasets through providers and resolvers, making DataExpress ideal for multi-tier applications.

After data is provided to an application or a data module, you can view and work with the data locally in data-aware controls. You can store your data to local memory (MemoryStore) or to a local file (DataStore). When you want to save the changes back to your database, you must resolve the data.


Using DataStore instead of MemoryStore

By default, once data is loaded into a JBuilder component, it's stored in memory through the use of a MemoryStore. You can use alternate storage systems such as the JDataStore by setting the StorageDataSet object's store property. (Currently MemoryStore and DataStore/DataStoreConnection are the only implementations of the Store interface required by the store property.)

The main advantages of DataStore over MemoryStore are transaction semantics, SQL support, and persistence, which enable offline computing. A DataStore remembers the rows fetched in a table, even after the application terminates and restarts. In addition, you can increase the performance of any application with large StorageDataSets. StorageDataSets using MemoryStore have a small performance edge over DataStore for a small number of rows. JDataStore stores data and indexes in an extremely compact format, however. As the number of rows in a StorageDataSet increases, using a DataStore provides much better performance and requires considerably less memory than using a MemoryStore.

Whether the data's storage is MemoryStore or DataStore often doesn't affect how you work with a StorageDataSet or other data-aware controls connected to the StorageDataSet. Storing Java objects in columns, however, does require you to use Java serialization (java.io.Serializable). If this isn't possible, you can't use DataStore components and you should use the default in-memory storage mechanism.


Using a JDataStore with StorageDataSets

You cache and persist StorageDataSets in a JDataStore by setting the values of the three properties discussed in "Connecting to a JDataStore with StorageDataSet." Persisting data from a provider usually involves the two subclasses of StorageDataSet with predefined providers: QueryDataSet (which uses QueryProvider) and ProcedureDataSet (which uses ProcedureProvider).

To store and persist the data from one of those StorageDataSets in a JDataStore using the design tools,

  1. Select the application's Frame file and switch to design view.

  2. Add a DataStoreConnection component from the Data Express tab of the component palette to the component tree.

  3. In the Inspector, select the fileName property of the DataStoreConnection. Use the Browse button to display the Open dialog box and enter the JDataStore file name. Click Open.

  4. Select the QueryDataSet or ProcedureDataSet component in the component tree. (Note that you can also use a TableDataSet with its provider property set to a QueryProvider or ProcedureProvider.)

  5. In the Inspector, set the storeName property to the name you want to use for the table stream in the JDataStore.

  6. In the Inspector, set the store property of the StorageDataSet to the DataStoreConnection component.


Tutorial: Offline editing with JDataStore

This tutorial presents the steps for creating an application that uses a DataStore component to enable the offline editing of data. The server database is a sample JDataStore file, employee.jds, accessed through the JDataStore Server. Don't confuse this file with the JDataStore used for persistence. Locate the sample file before beginning. It's installed in samples/JDataStore/datastores.
  1. Start the JDataStore Server from the menu item Tools|JDataStore Server.

  2. Create a new application by selecting File|New from the menu and doubleclicking the Application icon. In the Application Wizard:

  3. Switch to design view for the newly created PersistFrame.java.

  4. Add a Database component from the Data Express tab to the component tree.

  5. Open the connection property editor for the Database component in the Inspector. Set the connection properties to the database, using the correct path to the sample employee.jds file in place of d:/jbuilder4 in the URL:

    Property name

    Setting

    Driver

    com.borland.datastore.jdbc.DataStoreDriver

    URL

    jdbc:borland:dsremote://localhost/d:/jbuilder4/samples/JDataStore/datastores/employee.jds

    Username

    <use any name>

    Password

    <leave blank>

  6. Click the Test Connection button to check that you've set the connection properties correctly. When the connection is successful, click OK.

  7. Add a DataStoreConnection component from the Data Express tab to the component tree.

    Adding a DataStoreConnection component writes an import statement for the datastore package to your code and adds the JDataStore library to your project properties if it wasn't already listed.

  8. Open the fileName property editor for the DataStoreConnection component. Type in the name for a new JDataStore file. Be sure to include the full path. You can use the Browse button to help. You don't have to specify a file extension because a JDataStore always has the extension .jds. Click OK.

    Note: The Designer automatically creates this JDataStore file for you when it's connected to the StorageDataSet so that the tools work fully. When you run the application, the JDataStore file will already be there. But if you run the application on another computer, the JDataStore file won't be there. You will have to add extra code to create the JDataStore file if necessary as shown in "Creating a JDataStore file."

  9. Add a QueryDataSet component from the Data Express tab to the component tree.

  10. Open the query property editor for the QueryDataSet component in the Inspector and set the following properties:

    Property name

    Value

    Database

    database1

    SQL Statement

    select * from employee

  11. Click Test query to ensure that the query is runnable. When the gray area beneath the button indicates Success, click OK to close the dialog box.

  12. Set the storeName property of the QueryDataSet to employeeData.
  13. Set the store property to dataStoreConnection1 (the only choice).

  14. Add a JdbNavToolbar component from the dbSwing tab to the North position of the frame. Set its dataSet property to queryDataSet1.

  15. Add a JdbStatusLabel component from the dbSwing tab to the South position of the frame. Set its dataSet property to queryDataSet1.

  16. Add a TableScrollPane component from the dbSwing tab to the Center position of the frame.

  17. Add a JdbTable component from the dbSwing tab to the TableScrollPane. Set its dataSet property to queryDataSet1.

  18. Instead of adding code to call DataStore.shutdown() before exiting the application (as you did in an earlier tutorial), you can use a DBDisposeMonitor component to close JDataStore files automatically when you close the frame.

  19. Add a DBDisposeMonitor component from the More dbSwing tab to the component tree. Set its dataAwareComponentContainer property to this.

  20. Run PersistApp.java.

In the running application, make some changes to the data and click the Post button on the navigator to save the changes to the JDataStore file (the persistence JDataStore specified in step 7). Changes are also saved to the file when you move off a row, just as they are with an in-memory data set (MemoryStore).

Note: A data set in a JDataStore can have tens or hundreds of thousands of rows. Handling that much data using an in-memory data set would greatly slow application performance.

Close the application and run it again. You see the data as you edited it in the previous run. This is very different from the behavior of an in-memory data set. If you want, you can exit the application, shut down the JDataStore Server, and run the application again. Without any connection to the SQL database, you can continue to view and edit data in the JDataStore. You'll find this especially useful if you want to work with data offline such as at home or on an airplane.

Understanding how JDataStore manages offline data

So far in the tutorial, nothing has been saved back to the SQL database on the server. On the JdbNavToolbar, there are several buttons:

Options set in the queryDescriptor also have an effect on how data is stored, saved, and refreshed. In the queryDescriptor in this example, the Execute Query Immediately When Opened option is selected. This option indicates how data is loaded into the JDataStore file when the application was first run. On subsequent runs, the execution of the query is suppressed, because the data set is found in the JDataStore file instead of on the server. As a result,

When the Execute Query Immediately When Opened option is selected, existing data can't be overwritten (unless you call the StorageDataSet.refresh() method explicitly). This means that you can safely close and reopen a data set to change property settings in either a MemoryStore or in a DataStore without losing editing changes.

Once you've got data in the JDataStore file, you can run this application and edit data whether the database server is available or not. When you are working offline, you have to remember not to click the navigator's Save or Refresh button. If you do, you'll get an exception because the attempt to connect will fail, but you won't lose any of the changes you have made.


Restructuring JDataStore StorageDataSets

The Column Designer in the JBuilder UI designer provides support for moving, deleting, and inserting columns. You can also change Column data types. As the data type of a Column component in a StorageDataSet that is bound to a JDataStore changes, type coercions occur when going from one type to another.

To activate the Column Designer,

  1. Select the Data Module file in the Navigation pane.

  2. Press the Design tab for your DataModule.

  3. Right-click a StorageDataSet in the JBuilder Structure pane.

  4. Select Activate Designer.

The Column Designer works for StorageDataSets that are using a MemoryStore or a DataStore. MemoryStore performs all operations instantly. When a Column data type is changed, MemoryStore doesn't convert data values to the new data type. The old values are lost.

JDataStore doesn't perform the move/insert/delete/change type operations on StorageDataSets immediately. The structural change is noted inside the JDataStore directory as a pending operation. The StorageDataSet.getNeedsRestructure() method returns true when there is a pending restructure operation. You can still use a StorageDataSet with pending structural changes:

To make a pending restructure operation take effect, click the Restructure toolbar button in the Column Designer. You can also force the restructure operation to happen with code by calling the StorageDataSet.restructure() method.

You can use the restructure() method even when there are no pending structural changes to repair or compact a StorageDataSet and its associated indexes. (See the DataStoreConnection.copyStreams() method for another way of repairing damaged streams.)

Data type coercions

When the data type of a Column component in a StorageDataSet that is bound to a JDataStore is changed, type coercions occur when going from one type to another. The following table describes what happens when a data type is coerced to another data type. The data types on the left indicate the original data type of the Column with the data types listed along the top of the table indicating the new data type of the Column.

From\To

Big Decimal

Double

Float

Long

int

Short

boolean

Time

Date

Time stamp

String

Input Stream

Object

Big
Decimal

None

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

Double

Prec

None

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

Float

Prec

OK

None

Prec

Prec

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

Long

OK

Prec

Prec

None

Prec

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

int

OK

OK

Prec

OK

None

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

Short

OK

OK

OK

OK

OK

None

Prec

Prec

Prec

Prec

OK

Loss

Loss

boolean

OK

OK

OK

OK

OK

OK

None

Prec

Prec

Prec

OK

Loss

Loss

Time

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

OK

Loss

Loss

Date

Prec

Prec

Prec

Prec

Prec

Prec

Prec

None

None

Prec

OK

Loss

Loss

Time
stamp

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

Prec

None

OK

Loss

Loss

Here is a legend to the table:

Persistent column editing

Many DataSets, such as QueryDataSet and ProcedureDataSet, obtain their structure and row data from a SQL server. But sometimes you need to add columns to these DataSets or to create standalone tables. You can do this with the JBuilder Column Designer. Use it to change, delete, move, and add DataSet columns instantly at design time. The JDataStore uses a mapping table to provide an illusion that each structural operation has occurred. When you've made all the structural changes, click Restructure and the restructure occurs.

When the Designer is activated while a StorageDataSet node is selected, the columns for that node are displayed in a grid on the design surface. A toolbar for adding, deleting, moving up, moving down, and restructuring the data set appears above the grid:

Understanding structure changes

Regardless of whether a JDataStore is used, columns provided by a query or procedure must be merged with columns defined in code. When a JDataStore is being used, this merge still must happen, followed by a merge between the resulting column list and the columns found in the data set in the store.

Ideally, columns in the JDataStore are the same as those provided by the merge. But when you are developing an application, you might make changes. Even production applications change occasionally. You can continue to use a data set in a JDataStore when the columns in the JDataStore aren't the same as those from the application/provider merge. The provide operation automatically restructures the table after merging.

If you change a data set's structure in the Designer instead of editing the source code, the JDataStore tracks the changes. This makes the merge between application/provider columns and JDataStore columns easier. For instance, if you change the name of a calculated column through the Designer, you can continue to display and edit values in the column, because the JDataStore can map the old column name to the new one. If you make the same change in code, the JDataStore can only determine that one column was deleted and another was added. The JDataStore won't recognize that the column was renamed. So the deleted column won't be displayed and the added column will be empty and non-editable.

The following list provides more information for different types of data set structure changes:

Restructuring also packs the dataset and deletes indexes. The indexes are rebuilt when its needed.