Feature/csvstream - #115
Conversation
There was a problem hiding this comment.
IMHO, the current implementation does not meet the goal of the class/methods that were defined in the design document.
Blackboard: Team_5_GroupProjectProposal_2025_as_submitted.docx
| @@ -0,0 +1,1001 @@ | |||
| id,name,email | |||
There was a problem hiding this comment.
Do you need to push this output file to the repo?
It does not look related or required.
|
|
||
| IntStream.rangeClosed(1, 1000).forEach(i -> { | ||
| try { | ||
| String line = i + ",User_" + i + ",user" + i + "@mail.com"; |
There was a problem hiding this comment.
Why hard code it?
The stream could process anything, not only this use case.
| writer.write("id,name,email"); | ||
| writer.newLine(); | ||
|
|
||
| IntStream.rangeClosed(1, 1000).forEach(i -> { |
There was a problem hiding this comment.
We should not generate a large CSV file here.
The repository already has many sample CSV files at main/src/main/resources/sample-csvs
Check these:
- snakes_count_10000.csv
- hw_25000.csv
- mlb_players.csv
|
|
||
| import static org.junit.jupiter.api.Assertions.*; | ||
|
|
||
| class CsvStreamTest { |
There was a problem hiding this comment.
Good for the Start!
I suggest integrating your CsvStream class with the existing CSV library. Its main role is to be a lightweight wrapper around CsvReader and provide a Java Stream<Row> interface to make processing rows easy.
A minimal recommended API could look like this:
public final class CsvStream implements Closeable {
private CsvStream(CsvReader reader);
/** Wraps an existing CsvReader into a CsvStream */
public static CsvStream from(CsvReader reader);
/** Opens a CSV file with the given configuration */
public static CsvStream fromPath(Path path, CsvConfig config) throws Exception;
/** Opens a CSV file with configuration and explicit headers */
public static CsvStream fromPath(Path path, CsvConfig config, Headers headers) throws Exception;
/** Returns a Stream of Row objects for lazy processing */
public Stream<Row> stream();
/** Closes the underlying CsvReader */
public void close();
}How a user would use CsvStream:
Path file = Paths.get("data.csv");
CsvConfig config = CsvConfig.builder().setHasHeader(true).build();
try (CsvStream csvStream = CsvStream.fromPath(file, config)) {
csvStream.stream()
.filter(row -> row.get("age").asInt() > 30)
.forEach(row -> System.out.println(row.get("name")));
}
// CsvStream automatically closes the underlying CsvReaderKey Points for Implementation:
CsvStreamshould wrap aCsvReaderinternally.stream()should lazily read rows so you don’t load the entire file into memory.close()should release resources from the underlyingCsvReader.- Use
try-with-resourceswhen consumingCsvStreamto ensure proper cleanup.
b537c72 to
49fedc3
Compare
|
Working on it, to continue from @quadri00 version and implement @bogdansharp suggestions. |
- Wraps CsvReader for lazy, memory-efficient streaming - Implements Closeable for resource management - Factory methods: from(CsvReader), fromPath(Path, CsvConfig), fromPath(Path, CsvConfig, Headers) - Supports all Stream operations (filter, map, limit, forEach, count, collect) - 9 comprehensive unit tests covering all functionality - Extensive Javadoc with usage examples - Implements all suggestions from Bogdan's code review - All tests pass, build successful
da533d5 to
b43f6bd
Compare
|
Lots of commits too, let me squash it. |
|
Local build pass: |
csv stream puahed to main for review