Java I/O and Data Handling
Open in the Padhaiko app2.1 Streams and file handling
In Java, I/O (Input/Output) is modeled around the abstraction of a stream: an ordered sequence of data flowing either from a source into a program (input stream) or from a program to a destination (output stream). The classic java.io package separates raw binary streams from text character streams, while modern java.nio.file (NIO.2, introduced in Java 7) provides cleaner path and file utilities.
2.1.1 Byte and character streams
Byte Streams vs. Character Streams
- Byte Streams (
InputStream,OutputStream): Transfer raw 8-bit bytes (0to255). Used for binary data such as images, audio, PDFs, compiled.classfiles, and serialized objects.- Core implementations:
FileInputStream,FileOutputStream,BufferedInputStream,BufferedOutputStream,DataInputStream,DataOutputStream.
- Core implementations:
- Character Streams (
Reader,Writer): Transfer 16-bit Unicode characters (char), automatically converting between bytes on disk and UTF-16chars in memory using a character encoding (such as UTF-8). Used for plain text, CSV, XML, and JSON files.- Core implementations:
FileReader,FileWriter,BufferedReader,BufferedWriter,PrintWriter, and bridge classesInputStreamReader/OutputStreamWriter.
- Core implementations:
| Aspect | Byte Streams | Character Streams |
|---|---|---|
| Transfer Unit | 8-bit raw byte (int 0..255 or -1 EOF) | 16-bit Unicode character (char, -1 EOF) |
| Base Abstract Classes | java.io.InputStream, java.io.OutputStream | java.io.Reader, java.io.Writer |
| Encoding Awareness | None (reads/writes exact bytes untouched) | Encoding-aware (Charset, e.g., StandardCharsets.UTF_8) |
| Typical Data | Images, executables, compressed archives, network packets | Text files, logs, source code, CSV, JSON |
| Buffered Wrappers | BufferedInputStream, BufferedOutputStream | BufferedReader (readLine()), BufferedWriter (newLine()) |
- Why Buffering Matters: Unbuffered
FileInputStream.read()orFileReader.read()can invoke an OS system call on almost every byte/character. Wrapping withBufferedInputStreamorBufferedReader(default 8 KB internal buffer) reads large chunks in one OS call and serves individual reads from RAM, drastically improving I/O throughput.
2.1.2 Reading/writing text files
Using InputStreamReader / OutputStreamWriter (or Java 11+ FileReader(path, StandardCharsets.UTF_8)) guarantees consistent UTF-8 encoding across Windows, Linux, and macOS instead of relying on the OS default charset.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
public class TextFileCopy {
public static int copyWithLineNumbers(String src, String dest) throws IOException {
int lineCount = 0;
try (
BufferedReader reader = new BufferedReader(
new InputStreamReader(new FileInputStream(src), StandardCharsets.UTF_8));
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream(dest), StandardCharsets.UTF_8))
) {
String line;
while ((line = reader.readLine()) != null) {
lineCount++;
writer.write(lineCount + ": " + line);
writer.newLine();
}
}
return lineCount;
}
}2.1.3 Object streams and serialization
Serialization is the mechanism of converting an in-memory Java object graph into a portable byte stream so it can be saved to a file or transmitted over a network. Deserialization reconstructs the object from that byte stream.
Key Serialization Rules in Java
java.io.Serializable: A marker interface (contains no methods). Any class whose objects are serialized—and all non-transientfields referencing other objects—must implementSerializable, otherwiseNotSerializableExceptionis thrown at runtime.ObjectOutputStream&ObjectInputStream:writeObject(obj)serializes an object;readObject()deserializes and returnsObject(requiring a cast).serialVersionUID: Aprivate static final long serialVersionUID = 1L;field serving as a version stamp. If the class structure changes and theserialVersionUIDdiffers from the stored stream,InvalidClassExceptionis thrown.transientKeyword: Marks sensitive or derived fields (such as passwords, cached totals, or thread handles) to be skipped during serialization; upon deserialization,transientfields revert to their Java default values (null,0,false).staticFields: Belong to the class, not an instance, so they are not serialized with an object.
import java.io.*;
class UserSession implements Serializable {
private static final long serialVersionUID = 1L;
private String username;
private int loginCount;
private transient String sessionToken; // skipped during serialization
public UserSession(String username, int loginCount, String sessionToken) {
this.username = username;
this.loginCount = loginCount;
this.sessionToken = sessionToken;
}
@Override
public String toString() {
return username + " (logins=" + loginCount + ", token=" + sessionToken + ")";
}
}
public class SessionStorage {
public static void main(String[] args) throws IOException, ClassNotFoundException {
UserSession s = new UserSession("asha", 4, "SECRET-991");
try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("session.bin"))) {
out.writeObject(s);
}
try (ObjectInputStream in = new ObjectInputStream(new FileInputStream("session.bin"))) {
UserSession restored = (UserSession) in.readObject();
System.out.println(restored); // token is null after deserialization
}
}
}2.1.4 New I/O (NIO.2) file operations
Introduced in Java 7 (java.nio.file), NIO.2 replaces awkward java.io.File methods (which often returned booleans without explaining failures) with Path, Files, and rich exception reporting (NoSuchFileException, AccessDeniedException).
| Operation | NIO.2 Method | Behavior / Notes |
|---|---|---|
| Create Path | Path.of("data", "users.txt") | OS-independent path construction (Java 11+; Paths.get(...) in Java 7+) |
| Read Whole Text | Files.readString(path) | Reads entire UTF-8 file into a single String (Java 11+) |
| Read All Lines | Files.readAllLines(path) | Returns List<String> (for small/medium files) |
| Lazy Line Stream | Files.lines(path) | Returns Stream<String> (must close via try-with-resources) |
| Write Text | Files.writeString(path, text, StandardOpenOption.CREATE, StandardOpenOption.APPEND) | Writes/appends string cleanly |
| Copy / Move / Delete | Files.copy(src, dst, StandardCopyOption.REPLACE_EXISTING), Files.move(...), Files.deleteIfExists(p) | Throws descriptive IOException on failure |
| Directory Listing | Files.list(dir) / Files.walk(dir) | Non-recursive vs. recursive Stream<Path> traversal |
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
import java.util.List;
public class NioDemo {
public static void main(String[] args) throws IOException {
Path dir = Path.of("logs");
Files.createDirectories(dir);
Path logFile = dir.resolve("app.log");
Files.writeString(logFile, "INFO: Server started\n",
StandardOpenOption.CREATE, StandardOpenOption.APPEND);
List<String> lines = Files.readAllLines(logFile);
System.out.println("Total lines: " + lines.size());
}
}2.2 JSON and data Interchange
Modern distributed applications exchange structured data across programming languages and platforms using text-based interchange formats—primarily JSON (JavaScript Object Notation) and XML (eXtensible Markup Language).
2.2.1 Handling JSON data with libraries like Jackson or Gson
Java's standard library does not include a built-in JSON parser, so enterprise Java projects use Jackson (the default in Spring Boot) or Gson (by Google).
- Serialization (POJO -> JSON String): Converting a Java object or
recordinto JSON text. - Deserialization (JSON String -> POJO): Parsing JSON text back into a typed Java object or collection.
Using Jackson (com.fasterxml.jackson.databind.ObjectMapper)
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.List;
class Course {
@JsonProperty("course_code")
private String code;
private String title;
@JsonIgnore
private String internalNotes;
public Course() {} // default no-arg constructor needed by Jackson for POJOs
public Course(String code, String title, String internalNotes) {
this.code = code;
this.title = title;
this.internalNotes = internalNotes;
}
public String getCode() { return code; }
public String getTitle() { return title; }
public String getInternalNotes() { return internalNotes; }
}
public class JacksonDemo {
public static void main(String[] args) throws Exception {
ObjectMapper mapper = new ObjectMapper();
Course c = new Course("ENCT327", "Java Programming", "Draft");
String json = mapper.writeValueAsString(c);
// {"course_code":"ENCT327","title":"Java Programming"}
String listJson = "[{\"course_code\":\"ENCT327\",\"title\":\"Java Programming\"}]";
List<Course> courses = mapper.readValue(listJson, new TypeReference<List<Course>>() {});
System.out.println(courses.get(0).getTitle());
}
}Using Gson (com.google.gson.Gson)
Gson gson = new Gson();(ornew GsonBuilder().setPrettyPrinting().create()).String json = gson.toJson(obj);andCourse c = gson.fromJson(json, Course.class);.- Uses
@SerializedName("course_code")to customize JSON field names andTypeToken<List<Course>>(){}.getType()for generic collections (overcoming Java type erasure).
2.2.2 JSON vs. XML in REST APIs
| Aspect | JSON | XML |
|---|---|---|
| Syntax & Verbosity | Lightweight key-value pairs and arrays ({"id": 1}) | Verbose opening and closing tags (<id>1</id>) |
| Payload Size | Compact; lower bandwidth and faster serialization | Larger payloads due to repeated tag names |
| Native Data Types | String, number, boolean, null, array, object | Text-only by default (types require XSD schema) |
| Arrays | Native [...] syntax | Repeated child elements (<item>...</item>) |
| Schema & Validation | JSON Schema (lightweight) | Mature XSD (XML Schema Definition), DTD, XSLT |
| Browser / JS Support | Native (JSON.parse / JSON.stringify) | Requires DOM/SAX parser (DOMParser) |
| Primary Use Today | RESTful web APIs, microservices, mobile backends | Legacy SOAP services, enterprise B2B, pom.xml |
Worked Examples
2.1
A binary file of size 5,500 bytes is copied using a FileInputStream wrapped in a BufferedInputStream with a custom buffer array byte[] buf = new byte[2048]. How many read(buf) calls return positive byte counts, what is the byte count returned on each call, and what value signals End-of-File (EOF)?
Divide the total file size (5,500 bytes) sequentially by the buffer capacity (2,048 bytes):
- Call 1: reads bytes
0..2047-> returns2048bytes. Remaining:5500 - 2048 = 3452bytes. - Call 2: reads bytes
2048..4095-> returns2048bytes. Remaining:3452 - 2048 = 1404bytes. - Call 3: reads the remaining
1404bytes -> returns1404bytes. Remaining:0bytes.
On Call 4, no bytes remain in the stream, so in.read(buf) returns -1 (End-of-File sentinel), terminating the while ((bytesRead = in.read(buf)) != -1) loop. Notice why writing out.write(buf, 0, bytesRead) is essential: on Call 3, writing all 2048 bytes instead of 1404 would corrupt the output file with 644 stale bytes from Call 2.
There are 3 read(buf) calls returning positive counts: 2048, 2048, and 1404 bytes (2048 + 2048 + 1404 = 5500), followed by a 4th call returning -1 to signal EOF.
2.2
Compare the exact UTF-8 byte length of the following payload represented in compact JSON versus compact XML (without indentation/newlines):
- JSON:
{"id":101,"code":"ENCT327"} - XML:
<course><id>101</id><code>ENCT327</code></course>
Compute the byte count of each ASCII/UTF-8 string and the percentage reduction achieved by JSON over XML.
Count the exact ASCII characters (1 byte per ASCII character in UTF-8):
- JSON
{"id":101,"code":"ENCT327"}:{(1) +"id":101,(9) +"code":"ENCT327"(16) +}(1) = 27 bytes. - XML
<course><id>101</id><code>ENCT327</code></course>:<course>(8) +<id>101</id>(12) +<code>ENCT327</code>(20) +</course>(9) = 49 bytes.
Compute the byte savings and percentage reduction:
- Difference:
49 - 27 = 22bytes saved. - Percentage reduction:
(22 / 49) * 100 = 44.90%.
Compact JSON uses 27 bytes and compact XML uses 49 bytes, giving a 22-byte saving (44.90% smaller payload) for this single record.
2.3
How should a student answer the past exam question: "What is object serialization in Java? Explain the roles of Serializable, serialVersionUID, and transient with a short code example"?
Define Serialization and Deserialization Clearly: State that serialization converts an object's state into a byte stream (ObjectOutputStream.writeObject) for persistence or network transfer, and deserialization reconstructs the object (ObjectInputStream.readObject).
Explain the Three Key Mechanisms and Show Code:
Serializable: Marker interface indicating a class permits serialization (throwsNotSerializableExceptionotherwise).serialVersionUID: Version identifier checked during deserialization to ensure sender and receiver class definitions match (InvalidClassExceptionon mismatch).transient: Excludes sensitive/non-serializable fields (e.g.,transient String password) so they becomenull/0/falseon deserialization.
Include a 6–10 line class Account implements Serializable snippet showing serialVersionUID, a transient field, and try-with-resources.
A complete exam answer defines serialization/deserialization, states the roles of Serializable (marker), serialVersionUID (version compatibility), and transient (excluded fields reset to defaults), and provides a compact ObjectOutputStream / ObjectInputStream code example.
Exercises
Practice Exercises — Java I/O and Data Handling
Differentiate between Byte Streams and Character Streams in Java. Why should StandardCharsets.UTF_8 be passed explicitly when reading or writing text files?
Byte Streams vs. Character Streams:
- Byte streams (
InputStream,OutputStream) read and write raw 8-bit bytes without character encoding translation, making them suitable for binary files (images, audio,.class, serialized objects). - Character streams (
Reader,Writer) read and write 16-bit Unicode characters and perform byte-to-character encoding/decoding, making them suitable for text files.
Importance of Explicit UTF_8 Charset: Constructors like legacy new FileWriter("out.txt") use the operating system's default charset (which historically varied across Windows, Linux, and macOS). Specifying StandardCharsets.UTF_8 guarantees identical encoding and decoding across all environments and prevents corrupted non-ASCII characters.
Byte streams (InputStream/OutputStream) handle raw 8-bit binary data, whereas character streams (Reader/Writer) handle 16-bit Unicode text with charset conversion. Explicitly specifying StandardCharsets.UTF_8 avoids platform-dependent encoding bugs across operating systems.
A binary stream of 9,000 bytes is read in a loop using byte[] buffer = new byte[4096]. How many full 4,096-byte chunks are read, how many bytes are returned on the final non-empty read, and why must out.write(buffer, 0, n) be used instead of out.write(buffer)?
Calculate Chunk Counts:
- Integer division:
9000 / 4096 = 2full reads (2 * 4096 = 8192bytes). - Remainder for the 3rd read:
9000 - 8192 = 808bytes. - The 4th read returns
-1(EOF).
Why out.write(buffer, 0, n) is Required: read(buffer) overwrites only indices 0 through n - 1 (0..807 on the 3rd read). Indices 808..4095 still hold leftover bytes from the 2nd read. Calling out.write(buffer) would write 4096 - 808 = 3288 extra garbage bytes to the destination file.
2 full 4,096-byte chunks are read, followed by a final read of 808 bytes (then -1 at EOF). out.write(buffer, 0, n) ensures only the n newly read bytes are written, preventing leftover bytes from corrupting the output.
Explain the behavior of static and transient fields during Java object serialization and deserialization. What happens if a serialized class contains a non-Serializable non-transient field?
transient and static Field Behavior:
transientfields are excluded from the serialized byte stream. When the object is deserialized,transientfields are initialized to their type defaults (0forint,0.0fordouble,falseforboolean,nullfor object references). Note that constructors are not executed during standard deserialization.staticfields belong to the class rather than any object instance, soObjectOutputStreamnever writes them; after deserialization, astaticfield holds whatever value the class currently has in the receiving JVM.
Non-Serializable Reference Field: If an object has a non-transient, non-null reference to an object whose class does not implement Serializable, ObjectOutputStream.writeObject() fails at runtime by throwing java.io.NotSerializableException.
transient fields are omitted from the byte stream and revert to default values (null/0/false) upon deserialization; static fields are class-level and are not serialized. A non-transient reference to a non-Serializable object throws NotSerializableException at runtime.
Write a short NIO.2 Java snippet using Path and Files that creates a directory reports, writes "ENCT327 Pass" to reports/summary.txt (creating or replacing the file), and reads it back as a String. Also explain how Files.lines(path) differs from Files.readAllLines(path).
NIO.2 Implementation:
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
public class ReportIO {
public static String saveAndRead() throws IOException {
Path dir = Path.of("reports");
Files.createDirectories(dir);
Path file = dir.resolve("summary.txt");
Files.writeString(file, "ENCT327 Pass");
return Files.readString(file);
}
}Files.lines() vs. Files.readAllLines():
Files.readAllLines(path)eagerly loads every line into aList<String>in heap memory; large files can triggerOutOfMemoryError.Files.lines(path)returns a lazyStream<String>that reads lines on demand as the stream pipeline processes them, making it suitable for large log files (must be closed insidetry-with-resources).
Use Path.of, Files.createDirectories, Files.writeString, and Files.readString. Files.readAllLines loads the entire file into a List<String> in memory, whereas Files.lines streams lines lazily with minimal memory footprint.
Why is JSON generally preferred over XML in modern REST APIs? Compare Jackson (ObjectMapper) and Gson for JSON handling in Java, and calculate the percentage payload reduction if a JSON response is 32 bytes compared to an equivalent 58-byte XML response.
Why JSON Dominates REST APIs: JSON is less verbose (no closing tags), produces smaller payloads, natively represents numbers, booleans, null, and arrays, and parses directly in JavaScript (JSON.parse). XML remains common where strict XSD schema contracts or legacy SOAP integrations are required.
Jackson vs. Gson and Payload Calculation:
- Jackson: Default in Spring Boot; high-performance streaming + tree/databind (
ObjectMapper), rich annotations (@JsonProperty,@JsonIgnore), usesTypeReference<List<T>>for generic lists. - Gson: Lightweight Google library (
Gson/GsonBuilder), simpletoJson/fromJson, uses@SerializedNameandTypeToken<List<T>>. - Payload reduction:
(58 - 32) / 58 * 100 = 26 / 58 * 100 = 44.83%.
JSON is preferred in REST APIs for its compact syntax, native data types, smaller payloads, and direct browser compatibility; Jackson (ObjectMapper) is Spring Boot's high-throughput default while Gson offers a lightweight alternative. Reducing 58 bytes to 32 bytes saves 26 bytes, a 44.83% payload reduction.