Hi
In October 2025, @superthin asked, “Cần trợ giúp việc chuyển file không chờ upload xong qua Internet” I thought this would be a simple approach. Transferring files from point A to point B using P2P communication software is the key to accomplishing this task. For example, Linux’s fpt command enables file transfer. However, according to @superthin, the issue is transferring large amounts of data (large files), which can be several GB in size. There are numerous free and paid software programs available that can fulfill this need. I haven’t used any of the available software or Linux FPT. Upon closer inspection, free online file transfer programs limit the data size to 5 GB. Beyond that, you have to pay or subscribe.
Back to the root of the matter. The problems with file transfer limit not only the file size, but also the network packet size (due to the legacy: 4 bytes of int for files and 2 bytes of short int for data packets). This means that files larger than 4 GB require special processing before they can be sent over the network in 64 KB data packets. This physical limitation affects the transfer time. Added to this is the network load, which is usually unpredictable on the internet (regardless of VPN) – since your network is based on leased lines. Furthermore, the internet requires security, which requires additional effort for encryption and decryption and also impacts transfer times. To reduce the number of data packets—and thus the transfer time—most file transfer programs utilize data compression and decompression. This is advantageous for large files of more than a few GB, but not optimal for small files of a few MB.
Nowadays, software developers rarely program in assembly or C, but rather in OOP languages such as C++, JAVA, PYTHON, etc. This is due to the availability of low-level networking APIs such as Socket, UDP, etc. in all OOP languages. These APIs are reliable and significantly reduce development effort. The downside is that some IT developers have less and less knowledge of communication technology. To understand peer-to-peer (P2P), you need to have a thorough understanding of the communication flow between two parties (peers) and the processing order within the communication software and the operating system. It’s all about synchronization. And understanding synchronization is a delicate matter: not only between two peers, but also between executing code, such as threads within the software.
This code snippet illustrates the problems caused by a lack of operating system knowledge: When the SEND button is clicked, the Lambda expression code executes, and the remote server starts an Xfer thread to accept the incoming data using the retrieveFile() method. The Xfer thread runs in a thread pool (ExecutorService).
SEND.addActionListener(e -> {
...
String fileName = tFile.getText();
if (fileName.startsWith("$HOME")) fileName = fileName.replace("$HOME", uHome);
if (!(new File(fileName)).exists()) msg.setText(fileName+" is unknown.");
else try {
...
SocketChannel soc = SocketChannel.open(new InetSocketAddress(tIP.getText(), rPort));
soc.socket().setSendBufferSize(65536);
soc.socket().setTcpNoDelay(true);
// sendToken + fileName
byte[] buf = (token[0]+""+ZIP+fileName.substring(fileName.lastIndexOf(File.separator)+1)).getBytes("UTF-8");
soc.write(ByteBuffer.wrap(buf, 0, buf.length));
ByteBuffer bbuf = ByteBuffer.allocate(256);
int n = soc.read(bbuf); // wait for Reply
sendFile(soc, fileName, zip);
} catch (Exception ex) {
ex.printStackTrace();
}
});
...
private void sendFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
...
}
private String retrieveFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
...
}
...
private class Xfer implements Runnable {
public Xfer(SocketChannel soc) {
this.soc = soc;
}
private SocketChannel soc;
public void run() {
try {
...
if (buf[0] == (byte)token[0]) { // send from remote
soc.write(ByteBuffer.wrap(new byte[] { (byte)0x00 }, 0, 1));
retrieveFile(soc, fileName, zip);
} else if (buf[0] == (byte)token[1]) { // retrieve from remote
if (fileName.startsWith("$HOME")) fileName = fileName.replace("$HOME", uHome);
if (!(new File(fileName)).exists()) {
...
return;
}
sendFile(soc, fileName, zip);
} else { // Illegal. Do NOTHING
...
return;
}
} catch (Exception ex) { }
}
}This means that the Lambda expression code runs interactively and the Xfer code runs in the background. Interactive tasks always have higher priority than background tasks, and the local-remote code encounters synchronization issues between sendFile() and retrieveFile() on the remote server. The result: unpredictable or corrupted data on the remote site.
Priorities are managed by operating systems designed to prioritize interactive tasks to ensure a responsive user experience, while background tasks are executed with less urgency and do not interfere with foreground activities. This means that to reduce response time, the interactive lambda expression is served preferentially, so some sent data packets are lost to the remote thread running in the background. The synchronization issues can be easily solved if the code within the lambda expression, like the Xfer thread, is also placed in the thread pool to run in the background.
SEND.addActionListener(e -> {
...
// running in Background in thread pool
pool.execute(() -> {
String fileName = tFile.getText();
if (fileName.startsWith("$HOME")) fileName = fileName.replace("$HOME", uHome);
if (!(new File(fileName)).exists()) msg.setText(fileName+" is unknown.");
else try {
...
SocketChannel soc = SocketChannel.open(new InetSocketAddress(tIP.getText(), rPort));
soc.socket().setSendBufferSize(65536);
soc.socket().setTcpNoDelay(true);
// sendToken + fileName
byte[] buf = (token[0]+""+ZIP+fileName.substring(fileName.lastIndexOf(File.separator)+1)).getBytes("UTF-8");
soc.write(ByteBuffer.wrap(buf, 0, buf.length));
ByteBuffer bbuf = ByteBuffer.allocate(256);
int n = soc.read(bbuf); // wait for Reply
sendFile(soc, fileName, zip);
} catch (Exception ex) {
ex.printStackTrace();
}
});
});
...Encryption/decryption and compression/decompression work similarly. Therefore, I’ll show you how to work with compression/decompression in pure JAVA. If you search the internet for problems sending/receiving compressed/decompressed data over a socket, you won’t find any good advice or solutions (even on Stack Overflow). This JavaTechniques blog “Compressing Data Sent Over a Socket” is good and helpful if you need two generic CompressedBlockOutputStreams and CompressedBlockInputStreams for your own API library. However, I disagree with this statement:
The stream classes in java.util.zip work well for working with compressed files and similarly bounded data. They are, however, less useful for compressing continuous transmissions over a socket connection.
but agree with this
This is unfortunate, as the tradeoff between processor cycles (needed for compressing and decompressing) and bandwidth is often such that compression can have a considerable impact on performance of network applications
I’ll show you how to use GZIPInputStream and GZIPOutputStream with socket communication.
private void sendFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
byte[] buf = new byte[65536];
try {
int p;
ByteBuffer bbuf = ByteBuffer.allocate(65400);
FileChannel fc = (new FileInputStream(fileName)).getChannel();
GZIPOutputStream gzip = new GZIPOutputStream(soc.socket().getOutputStream(), true);
while ((p = fc.read(bbuf)) > 0) {
((ByteBuffer)bbuf.flip()).get(buf, 0, p);
gzip.write(buf, 0, p); // write to socket
gzip.flush();
bbuf.clear();
}
gzip.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
...
private void retrieveFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
int p;
GZIPInputStream gzip = null;
byte[] buf = new byte[65536];
GZIPInputStream gzip = new GZIPInputStream(soc.socket().getInputStream());
FileChannel fc = (new FileOutputStream(fileName, false)).getChannel();
while ((p = gzip.read(buf)) > 0) { // read from Socket
fc.write(ByteBuffer.wrap(buf, 0, p));
}
gzip.close();
}You can use JAVA Inflater and Deflater directly for decompression and compression. This significantly improves performance compared to using a generic compression/decompression stream. For example:
private void sendFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
int p;
ByteBuffer bbuf = ByteBuffer.allocate(65400);
FileChannel fc = (new FileInputStream(fileName)).getChannel();
byte[] deBuf = new byte[65536];
byte[] buf = new byte[65536];
Deflater deflater = new Deflater(Deflater.BEST_COMPRESSION);
deflater.setStrategy(Deflater.DEFAULT_STRATEGY);
while ((p = fc.read(bbuf)) > 0) {
((ByteBuffer)bbuf.flip()).get(buf, 0, p);
//compress by deflating the content
deflater.setInput(buf, 0, p);
deflater.finish();
int size = deflater.deflate(deBuf);
byte[] cBuf = new byte[8+size];
// compressed Length
cBuf[0] = (byte)((size >> 24) & 0xFF);
cBuf[1] = (byte)((size >> 16) & 0xFF);
cBuf[2] = (byte)((size >> 8) & 0xFF);
cBuf[3] = (byte)( size & 0xFF);
// original Length
cBuf[4] = (byte)((p >> 24) & 0xFF);
cBuf[5] = (byte)((p >> 16) & 0xFF);
cBuf[6] = (byte)((p >> 8) & 0xFF);
cBuf[7] = (byte)( p & 0xFF);
System.arraycopy(deBuf, 0, cBuf, 8, size);
soc.write(ByteBuffer.wrap(cBuf, 0, cBuf.length));
deflater.reset();
bbuf.clear();
}
fc.close();
}
private void retrieveFile(SocketChannel soc, String fileName, boolean zip) throws Exception {
int p;
byte[] dbuf, buf = new byte[65536];
ByteBuffer bbuf = ByteBuffer.allocate(65536);
FileChannel fc = (new FileOutputStream(fileName, false)).getChannel();
Inflater inflater = new Inflater();
while ((p = soc.read(bbuf)) > 0) {
((ByteBuffer)bbuf.flip()).get(buf, 0, p);
// length of the compressed block
int iLen = (buf[0] & 0xFF) << 24 |
(buf[1] & 0xFF) << 16 |
(buf[2] & 0xFF) << 8 |
buf[3] & 0xFF;
// length of decompress block
int oLen = (buf[4] & 0xFF) << 24 |
(buf[5] & 0xFF) << 16 |
(buf[6] & 0xFF) << 8 |
buf[7] & 0xFF;
dbuf = new byte[oLen];
//
inflater.reset();
inflater.setInput(buf, 8, iLen);
inflater.inflate(dbuf);
fc.write(ByteBuffer.wrap(dbuf, 0, oLen));
bbuf.clear();
}
fc.close();
}The following image gives an idea of the performance impact between In/Deflater and GZIPIn/GZIPOutputStream:
69.16 milliseconds versus 217.26 milliseconds.
The app is protected by a send/retrieve token between 1 and 255, defined and agreed upon between the owners. Note: It is recommended to use compression mode only if the file is REALLY larger than some MB.
As mentioned in my comment on @supethin’s post “Cần trợ giúp việc chuyển file không chờ upload xong qua Internet” the IPv6 internet address does NOT require port forwarding. The only issue is the long hex value, the IPv6 number that must be specified as the remote IP. If anyone is interested in XFileTransfer and/or ZFileTransfer, please leave a request in my DNH inbox.
83% thành viên diễn đàn không hỏi bài tập, còn bạn thì sao?