NanoDB - NanoDBManager and NanoDBServer last Part
This is the last part of my introduction to Object Oriented Database Implementation* and I hope I have given you an idea how to develop an OODB yourself. The package is royalty free. And the documentation can be found in 4 DNH NanoDB-blogs. You will have to copy (or download) the JAVA NanoDB package and examples yourself.
In a multi-user environment, you need to take care of each user’s activities so that they don’t get in each other’s way. Synchronization or locking mechanisms are just a thread-level tool, not task-level. This means that a user can come unexpectedly or leave suddenly (task), while a thread is usually independent but lives “together” in sync with its parent app. Therefore, we need an administrator or manager to monitor the users and take care of their potential conflicts. In our case, the users are client apps coming through NanoDBConnect-NanoDBWorker and the conflict area is the pool of NanoDB files opened by multiple users and the manager here is NanoDBManager. This API manages all opened NanoDB files and monitors their active status, if which NanoDB files are currently being closed and by whom and how (e.g. abort), so in that case it can launch an unlocking action for the keys locked by that user. A post-cleanup action if you will.
NanoDBManager API. The returned byte arrays (see figure in Part III) start with the first byte, which is 0x00 for a successful or 0x01 for a failed operation. The following bytes then depend on the operation, whether it is 1 byte (Boolean, 0x00: true, 0x01: false) or n bytes for a list of keys or a (non)serialized object or an exception string.
package nanodb;
import java.io.*;
import java.util.*;
import java.util.concurrent.*;
import java.nio.charset.Charset;
/**
NanoDBManager manages and synchronizes all NanoDBWorkers from accessing the underlying NanoDB files.
@author Joe T. Schwarz (c)
*/
public class NanoDBManager {
// for NanoDBWorker loop
public volatile boolean closed = false;
/**
contructor.
@param path String, directory path of NanoDB files
*/
public NanoDBManager(String path) {
this.path = path+File.separator;
}
/**
open - upper layer of NanoDB's open()
@param userID String
@param dbName String, NanoDB's name
@param charsetName String
@return byte array where the first byte element signifies the success (0) or failed (1), then a data in bytes
*/
public byte[] open(String userID, String dbName, String charsetName) {
try {
if (!usersList.contains(userID)) {
List<NanoDB> list = usersMap.get(userID);
if (list == null) list = Collections.synchronizedList(new ArrayList<>());
NanoDB nano = nanoMap.get(dbName); // already opened?
if (nano == null) { // new NanoDB
nano = new NanoDB(path+dbName, charsetName);
nanoMap.put(dbName, nano);
list.add(nano);
nano.open();
}
usersList.add(userID);
usersMap.put(userID, list);
}
return (""+(char)0x00+userID).getBytes();
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
close - upper layer of NanoDB's close()
<br>In case of error, the return byte array contains the error message.
@param userID String
@param dbName String, NanoDB's name
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] close(String userID, String dbName) {
if (usersList.remove(userID)) try {
NanoDB nano = nanoMap.get(dbName);
nano.removeLockedKeys(userID); // remove all keyslocked by this user.
if (usersList.size() == 0) { // last user?
nanoMap.remove(dbName).close();
usersMap.remove(userID);
} else { // there're some users
List<NanoDB> list = usersMap.get(userID);
list.remove(nano);
usersMap.put(userID, list);
}
} catch (Exception ex) { // exception
return (""+(char)0x01+ex.toString()).getBytes();
}
return new byte[] { (byte)0x00 };
}
/**
disconnect .
<br>In case of error, the return byte array contains the error message.
@param userID String
@return byte array where the first byte element signifies the success (0) or failed (1), then a data in bytes
*/
public byte[] disconnect(String userID) {
if (usersList.contains(userID)) try {
List<NanoDB> list = usersMap.get(userID);
for (NanoDB nano:list) {
nano.removeLockedKeys(userID);
nano.close();
}
usersList.remove(userID);
usersMap.remove(userID);
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
return new byte[] { (byte)0x00 };
}
/**
getKeys - upper layer of NanoDB's getKeys().
<br>In case of error, the return byte array contains the error message.
@param userID String
@return byte array where the first byte element signifies the success (0) or failed (1), then a list of keys in bytes
*/
public byte[] getKeys(String dbName) {
try {
List<String> keys = nanoMap.get(dbName).getKeys();
if (keys.size() == 0) return new byte[] { (byte)0x00, (byte)0x00, (byte)0x00 };
ByteArrayOutputStream bao = new ByteArrayOutputStream();
bao.write(new byte[] { (byte)0x00 }); // successfull
for (String k : keys) if (k.length() > 0) { // keyLength - keyContent as bytes
bao.write(new byte[] { (byte)((k.length() & 0xFF00) >> 8), (byte)(k.length() & 0xFF) });
bao.write(k.getBytes());
}
bao.flush();
bao.close();
return bao.toByteArray();
} catch (Exception ex) {
ex.printStackTrace();
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
autoCommit - upper layer of NanoDB's autoCommit().
@param dbName String
@param boo boolean, true: set, false: reset
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] autoCommit(String dbName, boolean boo) {
nanoMap.get(dbName).autoCommit(boo);
return new byte[] { (byte)0x00 };
}
/**
isAutoCommit - upper layer of NanoDB's autoCommit().
<br>The returned string is either "true" or "false" in lower case.
@param dbName String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] isAutoCommit(String dbName) {
if (nanoMap.get(dbName).isAutoCommit()) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
lock - upper layer of NanoDB's lock(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param userID String
@param dbName String, NanoDB's name
@param charsetName String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] lock(String userID, String dbName, String key) {
if (nanoMap.get(dbName).lock(userID, key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, 0x01 };
}
/**
unlock - upper layer of NanoDB's unlock(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param userID String
@param dbName String, NanoDB's name
@param charsetName String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] unlock(String userID, String dbName, String key) {
if (nanoMap.get(dbName).unlock(userID, key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
isLocked - upper layer of NanoDB's isLocked(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] isLocked(String dbName, String key) {
if (nanoMap.get(dbName).isLocked(key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
isExisted - upper layer of NanoDB's isExisted(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] isExisted(String dbName, String key) {
if (nanoMap.get(dbName).isExisted(key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
isKeyDeleted - upper layer of NanoDB's isKeyDeleted().
<br>The returned string is either "true" or "false" in lower case.
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] isKeyDeleted(String dbName, String key) {
if (nanoMap.get(dbName).isKeyDeleted(key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
readObject - upper layer of NanoDB's getObject().
<br>The returned is either the data or an error-message in byte array
@param userID String
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] readObject(String userID, String dbName, String key) {
try {
byte[] buf = nanoMap.get(dbName).readObject(userID, key);
byte[] bb = new byte[buf.length+1];
bb[0] = (byte)0x00;
System.arraycopy(buf, 0, bb, 1, buf.length);
return bb;
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
addObject - upper layer of NanoDB's addObject().
<br>In case of error, the return byte array contains the error message.
@param userID String
@param dbName String
@param key String
@param buf byte array of (non)serialized object
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] addObject(String userID, String dbName, String key, byte[] buf) {
try {
nanoMap.get(dbName).addObject(userID, key, buf);
return new byte[] { (byte)0x00 };
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
deleteObject - upper layer of NanoDB's deleteObject().
<br>In case of error, the return byte array contains the error message.
@param userID String
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] deleteObject(String userID, String dbName, String key) {
try {
nanoMap.get(dbName).deleteObject(userID, key);
return new byte[] { (byte)0x00 };
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
updateObject - upper layer of NanoDB's updateObject().
<br>In case of error, the return byte array contains the error message.
@param userID String
@param dbName String
@param key String
@param buf byte array of (non)serialized object
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] updateObject(String userID, String dbName, String key, byte[] buf) {
try {
nanoMap.get(dbName).updateObject(userID, key, buf);
return new byte[] { (byte)0x00 };
} catch (Exception ex) {
return (""+(char)0x01+ex.toString()).getBytes();
}
}
/**
commit - upper layer of NanoDB's commit(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param userID String
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] commit(String userID, String dbName, String key) {
if (nanoMap.get(dbName).commit(userID, key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
commitAll - upper layer of NanoDB's commitAll().
@param userID String
@param dbName String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] commitAll(String userID, String dbName) {
nanoMap.get(dbName).commitAll(userID);
return new byte[] { (byte)0x00 };
}
/**
rollback - upper layer of NanoDB's rollack(). The returned byte is x00 (OK) or x01 (failed).
<br>The returned string is either "true" or "false" in lower case.
@param userID String
@param dbName String
@param key String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] rollback(String userID, String dbName, String key) {
if (nanoMap.get(dbName).rollback(userID, key)) return new byte[] { (byte)0x00, (byte)0x00 };
return new byte[] { (byte)0x00, (byte)0x01 };
}
/**
rollbackAll - upper layer of NanoDB's rollbackAll().
@param userID String
@param dbName String
@return byte array where the first byte element signifies the success (0) or failed (1)
*/
public byte[] rollbackAll(String userID, String dbName) {
nanoMap.get(dbName).rollbackAll(userID);
return new byte[] { (byte)0x00 };
}
//
private String path;
private ByteArrayOutputStream bao = new ByteArrayOutputStream(65536);
private ConcurrentHashMap<String, NanoDB> nanoMap = new ConcurrentHashMap<>();
private List<String> usersList = Collections.synchronizedList(new ArrayList<>());
private ConcurrentHashMap<String, List<NanoDB>> usersMap = new ConcurrentHashMap<>();
}As you can see, the NanoDB package is small, neat, 100% pure JAVA, comprehensible and easy to implement. It has only four APIs:
- NanoDB: the core, an interface between logical and physical data using JAVA RandomAccessFile.
- NanoDBConnect: an interface for the user’s app.
- NanoDBWorker: an interface for NanoDBConnect.
- NanoDBManager: the management core of the NanoDB package.
To work with the four APIs, you need a server that serves the clients (NanoDBConnect), starts the service (NanoDBWorker) and runs the manager (NanoDBManager). All you need to do is implement a NanoDBServer in the GUI you want (JavaFX or SWING) and this server just needs to apply the NanoDB package (NanoDBWorker and NanoDBManager). I’ll show you an implementation of NanoDBServer below. It’s written in SWING, a SWING app. A bonus of NanoDBServer is the customized JPanel called SysMonSWING with the display of operating system activities: processing time, CPU usage, process usage, virtual memory, swap space and used memory.
NanoDBServer.java
package nanodb;
import java.awt.*;
import java.net.*;
import javax.swing.*;
import java.awt.event.*;
import java.util.concurrent.*;
//
import java.io.*;
import java.nio.channels.*;
/**
An implemented NanoDBServer using ServerSocketChannel.
@author Joe T. Schwarz (c)
*/
public class NanoDBServer extends JFrame {
//
public static void main(String... argv) throws Exception {
UIManager.setLookAndFeel("com.sun.java.swing.plaf.nimbus.NimbusLookAndFeel");
new NanoDBServer( );
}
/**
Constructor. This is the base for a customized NanoDB server
@param hostPort String, format: hostName:portNumber
@param path String, Directory path of NanoDB files
@exception Exception thrown by JAVA
*/
public NanoDBServer( ) throws Exception {
setTitle("NanoDBServer");
setDefaultCloseOperation(JFrame.DO_NOTHING_ON_CLOSE);
ExecutorService pool = Executors.newFixedThreadPool(1024);
//
JTextField jpath = new JTextField(System.getProperty("user.dir"));
JTextField jhost = new JTextField("localhost");
JTextField jport = new JTextField("9999");
JLabel lab = new JLabel("is NOT running");
jport.addKeyListener(new KeyAdapter() {
public void keyTyped(KeyEvent e) {
char c = e.getKeyChar();
if (c < '0' || c > '9') e.consume();
}
});
//
JButton start = new JButton("START");
start.addActionListener(a -> {
if ("EXIT".equals(start.getText())) {
nanoMgr.closed = true;
if (!running) {
running = false;
try {
dbSvr.close();
} catch (Exception ex) { }
}
pool.shutdownNow();
System.exit(0);
}
jhost.setEnabled(false);
jport.setEnabled(false);
jpath.setEnabled(false);
start.setText("EXIT");
pool.execute(() -> {
String host = jhost.getText().trim();
int port = Integer.parseInt(jport.getText());
jport.setEnabled(false);
boolean go = false;
try {
dbSvr = ServerSocketChannel.open();
dbSvr.socket().bind(new InetSocketAddress(host, port));
dbSvr.setOption(StandardSocketOptions.SO_RCVBUF, 65536);
jhost.setEnabled(false); jport.setEnabled(false);jpath.setEnabled(false);jLim.setEnabled(false);
//
go = true;
lab.setText("is running...");
nanoMgr = new NanoDBManager(jpath.getText().trim());
while (running) pool.execute(new NanoDBWorker(dbSvr.accept(), nanoMgr));
} catch (Exception e) { }
if (go) try {
dbSvr.close();
} catch (Exception ex) { }
pool.shutdownNow();
if (running) {
JOptionPane.showMessageDialog(this, "Cannot start NanoDBServer. Pls. check "+host+":"+port,
"ERROR", JOptionPane.ERROR_MESSAGE);
System.exit(0);
}
});
});
JPanel jptop = new JPanel();
jptop.add(new JLabel("HostName/IP")); jptop.add(jhost);
jptop.add(new JLabel("PortNumber")); jptop.add(jport);
jptop.add(lab);
JPanel jpSouth = new JPanel();
//
jpSouth.add(new JLabel("NanoDB Path")); jpSouth.add(jpath); jpSouth.add(start);
//
SysMonSWING sysmon = new SysMonSWING(600, 500);
pool.execute(sysmon); // start SystemMonitor
add("North", jptop);
add("Center", sysmon);
add("South", jpSouth);
getRootPane().setBorder(BorderFactory.createMatteBorder(5, 5, 5, 5, Color.LIGHT_GRAY));
setLocation(0, 0);
pack();
setVisible(true);
}
//
private NanoDBManager nanoMgr;
private volatile boolean running = true;
private ServerSocketChannel dbSvr = null;
}SysMonSWING.java is an extended JPanel.
package nanodb;
import java.awt.*;
import java.util.*;
import javax.swing.*;
import java.lang.management.*;
/**
System Moninor JPanel
@author Joe T. Schwarz (C)
*/
public class SysMonSWING extends JPanel implements Runnable {
/**
Constructor
@param width int, SysMonSWING Panel width
@param height int, SysMonSWING Panel height
*/
public SysMonSWING(int width, int height) {
super();
this.width = width;
this.height = height;
setPreferredSize(new Dimension(width, height));
// create the lists
ax = new ArrayList<Integer>(20);
mem = new ArrayList<Integer>(20);
pTi = new ArrayList<Integer>(20);
pLo = new ArrayList<Integer>(20);
cpu = new ArrayList<Integer>(20);
swp = new ArrayList<Integer>(20);
vir = new ArrayList<Integer>(20);
}
//
public void run() {
try {
com.sun.management.OperatingSystemMXBean osMaxBean =
(com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean();
long mem0, mem1, pTi0, pTi1, swp0, swp1, vir0, vir1;
int x0 = 0, fl = height - 235, fs = height - 290, ft = height - 290;
int fv = height - 100, fm = height - 145, fc = height - 185;
pTi0 = osMaxBean.getProcessCpuTime();
swp0 = osMaxBean.getFreeSwapSpaceSize();
mem0 = osMaxBean.getFreePhysicalMemorySize();
vir0 = osMaxBean.getCommittedVirtualMemorySize();
msg = "150 mSec. Period, "+osMaxBean.getAvailableProcessors()+
" cores, Total Mem. "+(osMaxBean.getTotalPhysicalMemorySize()/1048576)+
" GB, Horizontal Scale 2, Mem. Unit 1024 B";
while (true) {
x0 += 2; // scaling step
vir1 = osMaxBean.getCommittedVirtualMemorySize();
mem1 = osMaxBean.getFreePhysicalMemorySize();
swp1 = osMaxBean.getFreeSwapSpaceSize();
pTi1 = osMaxBean.getProcessCpuTime();
ax.add(x0);
swp.add(fs-(int)((swp0 - swp1)/1048576));
mem.add(fm-(int)((mem1 - mem0)/1048576));
vir.add(fv-(int)((vir1 - vir0)/1048576));
pTi.add(ft-(int)((pTi1 - pTi0)/1000000));
cpu.add(fc-(int)(osMaxBean.getSystemCpuLoad()*100));
pLo.add(fl-(int)(osMaxBean.getProcessCpuLoad()*100));
repaint();
java.util.concurrent.TimeUnit.MILLISECONDS.sleep(150);
if (x0 >= width) {
ax.clear();
mem.clear();
pTi.clear();
pLo.clear();
cpu.clear();
swp.clear();
vir.clear();
x0 = 0;
}
mem0 = mem1;
swp0 = swp1;
pTi0 = pTi1;
vir0 = vir1;
System.gc();
}
} catch (Exception e) { }
}
//
protected void paintComponent(Graphics g) {
g.setColor(Color.black);
g.fillRect(0, 0, width, height);
g.setColor(Color.white);
g.setFont(new Font("veranda", Font.BOLD,12));
g.drawString("NanoDB SERVER ACTIVITIES MONITORING", 150, 15);
int h = height-40;
g.setColor(Color.cyan);
g.drawLine(10, h, 30, h);
g.drawString("SwapSpace", 35, h);
g.setColor(Color.green);
g.drawLine(120, h, 140, h);
g.drawString("CPU-Load", 145, h);
g.setColor(Color.blue);
g.drawLine(230, h, 250, h);
g.drawString("Virtual Mem.", 255, h);
h = height-25;
g.setColor(Color.pink);
g.drawLine(230, h, 250, h);
g.drawString("ProcessTime", 255, h);
g.setColor(Color.red);
g.drawLine(120, h, 140, h);
g.drawString("ProcessLoad", 145, h);
g.setColor(Color.yellow);
g.drawLine(10, h, 30, h);
g.drawString("Used Memory", 35, h);
g.setColor(Color.white);
g.drawString(msg, 10, height-10);
for (int b, e, i = 0, j = 1, mx = pLo.size(); j < mx; ++i, ++j) {
b = ax.get(i);
e = ax.get(j);
g.setColor(Color.yellow);
g.drawLine(b, mem.get(i), e, mem.get(j));
g.setColor(Color.red);
g.drawLine(b, pLo.get(i), e, pLo.get(j));
g.setColor(Color.pink);
g.drawLine(b, pTi.get(i), e, pTi.get(j));
g.setColor(Color.green);
g.drawLine(b, cpu.get(i), e, cpu.get(j));
g.setColor(Color.cyan);
g.drawLine(b, swp.get(i), e, swp.get(j));
g.setColor(Color.blue);
g.drawLine(b, vir.get(i), e, vir.get(j));
}
}
//
private String msg = "";
private int width, height;
private ArrayList<Integer> ax, mem, pLo, pTi, cpu, swp, vir;
}
For simplicity, NanoDBServer does not provide client authentication (login) or logging functionality. However, as an IT developer, you can create your own server that provides these features and the limit size for NanoDB cache (currently set to two MB).
To run the NanoDBServer mentioned above, you need to compile the entire package and run the server as follows:
java -Xms2048m nanodb.NanoDBServer
// as nanodb.jar file with all API
java -Xms2048m -jar nanodb.jarNote: On Windows, you can run the Windows version of javaw without having to look for an EXE wrapper:
javaw -Xms2048m nanodb.NanoDBServer
// as nanodb.jar file with all API
javaw -Xms2048m -jar nanodb.jarHow to build nanodb.jar:
- create a manifest.m file as follows:
Manifest-Version: 1.0
Class-Path: nanodb.jar
Created-By: 1.0.0 (Joe T. Schwarz)and save it in your NanoDB subdirectory resources.
- create a batch file (or shell on linux) as follows:
REM compile to tmp
javac -g:none -d ./tmp *.java
cd tmp
REM build jar file
jar -cvfme ../nanodb.jar ../resources/manifest.mf nanodb.NanoDBServer nanodb/*.class > ../log.txt
cd ..
REM remove tmp
rmdir /s /q tmp
REM start nanodb.jar
javaw -Xms2048m -jar nanodb.jarand save it in your NanoDB directory.
- open a WINDOWS cmd and run it…
If you do this, the result is:
C:\JoeApp\ODB\NanoDB>buildjar
C:\JoeApp\ODB\NanoDB>REM compile to tmp
C:\JoeApp\ODB\NanoDB>javac -g:none -d ./tmp *.java
C:\JoeApp\ODB\NanoDB>cd tmp
C:\JoeApp\ODB\NanoDB\tmp>REM build jar file
C:\JoeApp\ODB\NanoDB\tmp>jar -cvfme ../nanodb.jar ../resources/manifest.mf nanodb.NanoDBServer nanodb/*.class 1>../log.txt
C:\JoeApp\ODB\NanoDB\tmp>cd ..
C:\JoeApp\ODB\NanoDB>REM remove tmp
C:\JoeApp\ODB\NanoDB>rmdir /s /q tmp
C:\JoeApp\ODB\NanoDB>REM start nanodb.jar
C:\JoeApp\ODB\NanoDB>javaw -Xms2048m -jar nanodb.jarClick START to start NanoDBServer, then open two or three CMD WINDOWS and start NanoDBClient from there and see how NanoDB works in a multi-user environment.
End of NanoDB final part.
The entire package can be found on GITHUB.
Comments and suggestions are welcome.
83% thành viên diễn đàn không hỏi bài tập, còn bạn thì sao?