Object Data Management System (ODMS) - The foundation of OO Database (OODB)
OODB
1
White

Joe viết ngày 02/07/2018

IT today is quite different to the old IT of the past. In the past business applications mostly dealt with "manageable" data. The data were managed by Data Management Systems (DMS) which could be "Relational" or "Hierarchical" or "Networked". The old business data are usually about accounting, general ledger, tax, inventory, payroll, etc. Today the information explodes. It balloons exponentially from day to day. We deal with Big Data. Very very big amount of Data.

Curiously is that we still toil and labor with the old way which makes our life more and more miserable. The reason is explainable: big data. BigData is object-oriented and object can be anything: besides the "old conventional" data BigData include images, music, sounds and any combitnation of (all of) them. Also, BigData is chaotic, messy and confusing.

Nevertheless we, the IT people, labor hard to "press" the big chaotic BigData into some "obsolete" mold which is beautified as either "relational" or "hierarchical" or (worst of all) "networked". And the most popular mold in use is the relational DMS model (known as SQL Databases). The result is disastrous, catastrophic. However, some efforts are on the way to remedy the "Object-phobic" problems of some IT developers: NoSQL, Jade, ObjectDB, etc.

The main problem of the IT users (and IT newbies in particular) is their IT-ignorance. They are either "laymen" (who're by chance into IT profession) or Mathematicians (who have to work with IT.) These IT dudes make the IT world becoming a hodge-podge world of dandies, nerds and fops. As usually they rarely understand the language spoken by Software men.

       +-------------------+---------------+-------------------+
       ! IT man            ! Mathematician ! IT User or Layman !
       +-------------------+---------------+-------------------+
       ! file              ! relation      ! table             !
       ! field             ! attribute     ! column            !
       ! record            ! tuple         ! row               !
       ! number of fields  ! cardinality   ! nunber of columns !
       ! number of records ! degree        ! nunber of rows    !
       ! list of files     ! Data Model    ! list of tables    !
       ! normalized        ! normalized    ! simplified        !
       ! non-repetitive    ! normalized    ! unique            !
       ! combine files     ! join/union    ! concatenate       !
       ! command           ! view          ! script            !
       ! Programming       ! Programming   ! coding            !
       ! ...               ! ...           ! ...               !
       +-------------------+---------------+-------------------+

As you see, it's the Tower of Babel. No wonder that IT software is the most miserable product at all. The misery is visible in form of bugs or incomplete functionalities.

Modern IT is Object-oriented. Business applications require presentable data (e.g. pie chart), visible data for a dynamic process (e.g animating stock chart), etc. All that can be only solved by Object-Oriented implementation. Relational "Atomization" always leads to defacing of object (or data). Object-entity (or Entity relationship) usually gets lost.

C++, C#, JAVA, PYTHON, etc. are OO programming tools. Some IT newbies still confuse business applications with procedural programs. The differences are technical and logic-profound. Business Applications require, for example, concurrency, parallelism, big data management, high performance and strict (data) consistency. And some of OOP tools are unsuited for such strict requirements. PYTHON is, for example, dynamic and good, but for business applications with concurrency, high performance and BigData management? Some PYTHON gurus may argue that there're enough (free) 3rd party libs which support the needed requirements. Yes, it's fine for private use. For a business any minor break could cause a big havoc or ruins the whole business (or the company itself.)

The popular C# and JAVA support "serialized" data. And I emphasize "Object Data Management System" or ODMS. I am not talking about any specific Object-Oriented Database (OODB), but about the possibility to use existing standard tools (i.e. APIs) to manage the objects and their BigData and to implement an ODMS. Further, it's impossible to cover this aspect in all details with examples in both C# and JAVA. Hence the examples are given in JAVA (reason: see programming "Taste".)

Serialization is a technique that allows OO developers to freeze and to save an object in its current state on any permanent medium (disk, CD, USB, etc.) The saving-retrieving mechanism is done by Object-IOs (ObjectInputStream and ObjectOutputStream or IFormatter with Stream). To do that objects must be implemented as serializable objects. C# is slightly different to JAVA.

C#

[Serializable]
public class MyObject {
  ...
}
...
MyObject obj = new MyObject();
...
IFormatter myFormatter = new BinaryFormatter();
Stream oos = new FileStream("C:\\objectdata.txt", FileMode.Create, FileAccess.Write);
myFormater.serialize(oos, obj);
oos.close();
...
Stream ois = new FileStream("C:\\objectdata.txt", FileMode.Open, FileAccess.Read);
MyObject obj = (MyObject)myFormatter.deserialize(ois);
... 

JAVA

public class MyObject implements Serializable {
  ...
}
...
MyObject obj = new MyObject();
...
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("C:\\objectdata.txt"));
oos.write(obj);
oos.close();
...
ObjectInputStream ois = new ObjectInputStream(new FileInputStream("C:\\objectdata.txt"));
MyObject obj = (MyObject)ois.readObject();
...

Serialization with JAVA is more straight-forward than with C#. JAVA distinguishes Object-streams between OUT and IN, while C# uses an IFormatter to do the serialization works. It depends on your "programming taste". And "taste" is like "religion". It cannot be discussed openly.

Object Data Management is not so complicated as one's used to think. Objects that need to be saved have only to be "marked" as "serializable". Then the "marked" objects can be written or read with ObjectOutputStream or ObjectInputStream. Simple, isn't it ?

As said, OO business applications require "tables", "rows" and "columns", too. How to do that an OODeveloper must be versed with OO. A row is a record. A Record is an object. Many rows mean many objects. A real file that contains many serialized objects needs to be handled differently if the objects are read CONSECUTIVELY. Otherwise it could cause a big headache to those who aren't versed with Object-Oriented programming.

import java.io.*;
public class Object_IO  implements Serializable {
    public static void main(String[] args) throws Exception  {
      Object_IO oio = new Object_IO();
      oio.runIO();
    }
    public void runIO() throws Exception {
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("ObjSer.txt"));
      for (int i = 0; i < 3; ++i) {
        ObjSer obj = new ObjSer();
        obj.key = "Key_"+i;
        obj.data = "Data_"+i;
        oos.writeObject(obj);
      }
      oos.flush();
      oos.close();
      ObjectInputStream ois = new ObjectInputStream(new FileInputStream("ObjSer.txt"));
      for (int i = 0; i < 3; ++i) {
        ObjSer obj =  (ObjSer) ois.readObject();
        System.out.println("Key:"+obj.key+" has "+obj.data);
      }
      ois.close();
    }
  }
 public class ObjSer implements Serializable {
      public String key, data;
}

This little app works perfectly. It prints out 3 lines.

C:\tmp>javac -g:none -d ./classes Object_IO.java

C:\tmp>java Object_IO
Key:Key_0 has Data_0
Key:Key_1 has Data_1
Key:Key_2 has Data_2

C:\tmp>

It won't work if the written objects are scattered between open and close (what is usually the case). This example shows you 3 objects were written scatteringly between closed-and-reopened.

import java.io.*;
public class Object_IOE  implements Serializable {
    public static void main(String[] args) throws Exception  {
      Object_IOE oio = new Object_IOE();
      oio.runIO();
    }
    public void runIO() throws Exception {
      IO(0, false); // false: create
      // do some other work then reopen
      IO(1, true);  // true: append
      // do some other work again and reopen
      IO(2, true);
      //
      ObjectInputStream ois = new ObjectInputStream(new FileInputStream("ObjSer.txt"));
      for (int i = 0; i < 3; ++i) {
        ObjSer obj =  (ObjSer) ois.readObject();
        System.out.println("Key:"+obj.key+" has "+obj.data);
      }
      ois.close();
    }
    // Serialized IO
    private void IO(int i, boolean append) throws Exception {
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("ObjSer.txt", append));
      ObjSer obj = new ObjSer();
      obj.key = "Key_"+i;
      obj.data = "Data_"+i;
      oos.writeObject(obj);
      oos.flush();
      oos.close();
    }
}

The headache is this exception StreamCorruptedException

C:\tmp>javac -g:none -d ./classes Object_IOE.java

C:\tmp>java Object_IOE
Key:Key_0 has Data_0
Exception in thread "main" java.io.StreamCorruptedException: invalid type code: AC
        at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.base/java.io.ObjectInputStream.readObject(Unknown Source)
        at Object_IOE.runIO(Unknown Source)
        at Object_IOE.main(Unknown Source)

C:\tmp>

The reason is inside the marking mechanism. ObjectOutputStream "marks" an ID as object-stream for the first object and all the subsequent followings. If the file is closed and reopened each new object is (re)marked again so that the next read includes the "marking ID" as a part of the reading object. The casting tries to rebuild an object from the serialized data (incl. the "marking ID") and fails. It interprets the whole chunk as corrupted data. Knowing that OODeveloper should be aware of such odd circumstance. To circumvent this oddness any experienced OODeveloper will (re)open the file separately before a read is executed.

import java.io.*;
public class Object_IOF  implements Serializable {
    public static void main(String[] args) throws Exception  {
      Object_IOF oio = new Object_IOF();
      oio.runIO();
    }
    public void runIO() throws Exception {
      IO(0, false);
      // do some other work
      IO(1, true);
      // do some other work again
      IO(2, true);
      // Separate the FileInputStream...
      FileInputStream fip = new FileInputStream("ObjSer.txt");
      //
      ObjectInputStream ois = null;
      for (int i = 0; i < 3; ++i) {
        // reopen again and again...
        ois = new ObjectInputStream(fip);
        ObjSer obj =  (ObjSer) ois.readObject();
        System.out.println("Key:"+obj.key+" has "+obj.data);
      }
      if (ois != null) ois.close();
    }
    private void IO(int i, boolean append) throws Exception {
      ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("ObjSer.txt", append));
      ObjSer obj = new ObjSer();
      obj.key = "Key_"+i;
      obj.data = "Data_"+i;
      oos.writeObject(obj);
      oos.flush();
      oos.close();
    }
}

The output is now correct

C:\tmp>javac -g:none -d ./classes Object_IOF.java

C:\tmp>java Object_IOF
Key:Key_0 has Data_0
Key:Key_1 has Data_1
Key:Key_2 has Data_2

C:\tmp>

With the possibility to stow more objects in a file you are now in a favorable position to build your own "Object-Oriented Database" which focuses only on the OOLanguage of your favor. Because of "BigData" and because of saving space you should compress the objects using the available GZIP APIs (JAVA: GZIPInputStream and GZIPOutputStream).

Saving/Retrieving data is the first step of ODMS. Assembling or sorting data is the advanced ODMS. JAVA and C# offer a lot of tools or APIs that deal with the sorting/comparing/manipulating of strings or of objects. It's feasible to create a table with some columns and rows (file/fields/records) with the aivailable APIs. The traditional part with Table-Column-Row is therefore easily to manage. The advanced part (consistency, multiuser, networking, etc.) has to be solved by OOP paradigm.

An Object in OOP is an active entity. It contains not only data (String, int, double, images, sound, etc.), but also alien objects and its own active parts (i.e. methods). Methods of serialized object are immediately active after retrieving. Object Data Management System ODMS is, like DMS, the foundation of Object-Oriented Database (OODB). I demonstrate hereunder an ODMS in the simplest implementation way. An ODMS must provide at least the following functions as the basics of a workable OODB:

  • Open (with auto-create)
  • Read/Write
  • query (of existing key)
  • Close

The multiuser (as server -with Networking), data consistency, thread-safe and concurrency are the advanced features of a full-fledged OODB. If I have time in the near future I show you how to implement a full-fledged OODB (what you have to do is to check KIPALOG frequently for the next ODMS-blog).

The Object Data Management System ODMS.java provides exactly the 5 mentioned methods.

  • Method open() checks for the existence of the given file and creates it if the file does not exist. You can improve open() with a boolean value to give the users 2 options, for example, true: create, false: exit. In case of "true" 2 files will be created: a keys-file (with preceding "Keys_"+fileName) and the serialized object file (fileName). In case of existed fileName all objects will be cached (HashMap) in order to accelerate the successive accesses.
  • Method close() clear the cache. No succesive Read/Write is possible
  • Method readObject(String key) provides the access of object with the given key.
  • Method writeObject(String key, Object obj) insert an object with the given key into OODB file. Existed object with the same key will be overwritten.
  • Method isExisted(String key) return true if key exists
import java.io.*;
import java.util.*;
// Joe Nartca (c)
public class ODMS {
  public ODMS(String fileName) {
    this.fileName = fileName;
    cache = new HashMap<String, Object>();
  }
  private String fileName;
  private boolean created = false;
  private HashMap<String, Object> cache;
  //
  private void create( ) throws Exception {
    try (FileOutputStream fk = new FileOutputStream("Keys_"+fileName, false);
         FileOutputStream fo = new FileOutputStream(fileName, false)) {
         fk.close();
         fo.close();
         return;
    } catch (Exception ex) { }
    throw new Exception("Unable to create OODB:"+fileName);
  }
  //
  public boolean isExisted(String key) {
    return cache.get(key) != null;
  }
  //
  public void writeObject(String key, Object obj) throws Exception {
    if (cache.containsKey(key)) return;
    try (FileOutputStream fk = new FileOutputStream("Keys_"+fileName, true);
         ObjectOutputStream oo = new ObjectOutputStream(new FileOutputStream(fileName, true))) {
      fk.write((key+"\n").getBytes());
      oo.writeObject(obj);
      cache.put(key, obj);
      fk.close();
      oo.close();
      return;
    } catch (Exception e) {e.printStackTrace();}
    throw new Exception("Unable to insert key:"+key+" or Object:"+obj.toString());
  }
  //
  public Object readObject(String key) throws Exception {
    Object obj = cache.get(key);
    if (obj == null) throw new Exception("Unknown key:"+key);
    return obj;
  }
  // dummy close
  public void close() { }
  //
  public void open( ) throws Exception {
    if (!(new File(fileName)).exists()) create();
    else try { // cache the objects
      FileInputStream fin = new FileInputStream("Keys_"+fileName);
      byte[] cont = new byte[fin.available()];
      int n = fin.read(cont);
      fin.close();
      if (n == 0) throw new Exception("Keys_"+fileName+" does not exist.");
      String[] keys = (new String(cont, 0, n)).replaceAll("\r","").split("\n");
      // load Cache
      fin = new FileInputStream(fileName);
      if (fin.available() == 0) throw new Exception(fileName+" is empty.");
      // cache the data
      try {
        ObjectInputStream ois = null;
        for (int i = 0; i < keys.length; ++i) {
          if (keys[i].length() > 0) {
            ois = new ObjectInputStream(fin);
            cache.put(keys[i], ois.readObject());
          }
        }
      } catch (Exception e) { }
      fin.close();
    } catch (Exception ex) {
      throw new Exception("Cannot open:"+fileName);
    }
  }
}

An application. Object: ObjSer.java and TestODMS.java

public class ObjSer implements java.io.Serializable {
    private static final long serialVersionUID = 1L;
    public String key, data;
    public ObjSer(String key, String data) {
      this.key = key;
      this.data = data;
    }
}

TestODMS.java

public class TestODMS {
    public static void main(String... args) throws Exception  { 
      ODMS odm = new ODMS(args.length == 0?"ObjSer.txt":args[0]);
      odm.open( ); // create DB
      for (int i = 0; i < 5; ++i) {
        odm.writeObject("Key_"+i, new ObjSer("Key_"+i, "Data_"+i));
      }
      System.out.println("You've created 5 keys: Key_0, Key_1, Key_2, Key_3, Key_4\n"+
                         "What key you want to retrieve? Any unknown key to exit.");
      byte[] line = new byte[80];
      int nl = System.getProperty("line.separator").length();
      while (true) {
        int n = System.in.read(line);
        String key = new String(line, 0, n-nl);
        if (key.length() > 0) {
          if (!odm.isExisted(key)) {
            System.out.println("Thank for playing with me. Bye Bye...");
            odm.close();
            return;
          }
          try {
            ObjSer obj = (ObjSer) odm.readObject(key);
            System.out.println("Read:"+obj.key+", "+obj.data);
          } catch (Exception ex) {
            ex.printStackTrace();
          }
        }
      }
    }
}

And here is the "result"

C:\odm>javac -g:none -d ./classes TestODMS.java

C:\odm>java TestODMS
You've created 5 keys: Key_0, Key_1, Key_2, Key_3, Key_4
What key you want to retrieve? Any unknown key to exit.
Key_3
Read:Key_3, Data_3
out
Thank for playing with me. Bye Bye...

C:\odm>

Another Example with invocations of 2 implemented methods toString() and show() of serialized object Animal.

// Joe Nartca (c)
public class Animal implements java.io.Serializable {
    private static final long serialVersionUID = 1L;
    private ImageIcon img;
    public String name, pic, weight;
    public Animal(String name, String pic, String weight) {
        this.pic = pic;
        this.name = name;
        this.weight = weight;
    }
    public void show(JFrame jf) {
        new Display(jf, getImage( ));
    }
    public String toString() {
        return "ZooAnimal:"+name+" has a weight "+weight+"Kg. Image@pic:"+pic;
    }
    private ImageIcon getImage() {
        try { // fetch image from the WEB
            return new ImageIcon(new URL(pic));
        } catch (Exception e) { }
        return null;
    }
    public void display(JFrame jf) {
        new Display(jf, getImage());
    }
    //
    private class Display extends JDialog {
        public Display(JFrame jf, ImageIcon icon) {
            super(jf, true);           
            setImage(jf, icon);
        }
        private void setImage(JFrame jf, ImageIcon img) { 
            setTitle("Animal:"+name);
            setDefaultCloseOperation(DISPOSE_ON_CLOSE);
            JButton but = new JButton(img);
            but.setContentAreaFilled(false);
            but.setBorderPainted(false);
            add(but, BorderLayout.CENTER);
            setSize(img.getIconWidth()+10, img.getIconHeight()+10);
            //
            setLocationRelativeTo(jf);
            setVisible(true);
        }
    }
}

the app

public class Zoo extends JFrame {
     ODMS odm = new ODMS("ZooAnimals.txt");
     ....
     but = new JButton("Animals");
     but.addActionListener(a->{
        String key = JOptionPane.showInputDialog(this,"AnimalName");
        if (key != null) {
            try {
               Animal ani = (Animal) odm.readObject(key);
               ta.append("\n-------Object is Animal------\n"+ani.toString());
               ani.show(this);
            } catch (Exception e) { }
         }
      });
      ...
      odm.close();
      ...
}

Here is the result:
ALT

Joe Nartca

Bình luận


White
{{ comment.user.name }}
Bỏ hay Hay
{{comment.like_count}}
Male avatar
{{ comment_error }}
Hủy
   

Hiển thị thử

Chỉnh sửa

White

Joe

28 bài viết.
207 người follow
Kipalog
{{userFollowed ? 'Following' : 'Follow'}}
Cùng một tác giả
White
27 11
Fuzzy Logic and Machine Learning Hi First of all: I apologize everyone for my writing in English. I come to this site because someone of Daynhauh...
Joe viết 12 tháng trước
27 11
White
25 11
You're a fresh graduate and work for more than 12 months in an IT company with some boring coding tasks... The tasks are unchallenging. Day in, day...
Joe viết 1 tháng trước
25 11
White
23 14
Chao cac ban To the Admins: if you think that this posting breaks some rules of your site please just delete it. NO need to send me a feedback. Th...
Joe viết 20 ngày trước
23 14
{{like_count}}

kipalog

{{ comment_count }}

bình luận

{{liked ? "Đã kipalog" : "Kipalog"}}


White
{{userFollowed ? 'Following' : 'Follow'}}
28 bài viết.
207 người follow

 Đầu mục bài viết

Vẫn còn nữa! x

Kipalog vẫn còn rất nhiều bài viết hay và chủ đề thú vị chờ bạn khám phá!