how to split a buffer ?

Question

Hellow i'm receiving a big Buffer object and i want to split it just like Array.split()
i'm using node js but if has a solution in another language, i will try to migrate it, thank!

Answer 1

If you have a string, split will work.
If your buffer is an Array, splice or slice will work. splice and slice is more like string's substring method, though.

There is no split method for Arrays in JavaScript.

Performing something vaguely similar to split with an Array wouldn't be too hard but you'd have to write the function yourself since it isn't built in.

Here is something a bit like an Array version of string's split method:

function mySplit(a, delimiter) {
var result = [];
var currentToken = [];
for (var i = 0; i < a.length; i++) {
if (a[i] === delimiter) {
if (currentToken.length !== 0)
result.push(currentToken);
currentToken = []; // clear array
}
else {
currentToken.push(a[i]);
}
}
if (currentToken.length !== 0)
result.push(currentToken);

return result;
}

var a = [1, 2, 4, -1, 6, 7, 9, 10, -1, 2, 6, 1, -1];

console.log(mySplit(a, -1));
// prints this in node.js:
// [ [ 1, 2, 4 ], [ 6, 7, 9, 10 ], [ 2, 6, 1 ] ]

// similar to how console.log('hello world how are you?'.split(' ')); prints
// [ 'hello', 'world', 'how', 'are', 'you?' ]

Answer 2

if I have a huge buffer like files worth 500 MB sent by a multipart form-data, this method will take a huge amount of time, and I'm not sure it will go through to the end

Answer 3

Instead of guessing that it possibly won't work, can you test it and see if and how it fails? I can't think of a reason for it to fail if you have a few gigabytes of RAM available for your nodejs application.

The mySplit function I wrote uses an optimal time complexity of O(n) so that can't be beat but if you see a specific problem with it while testing, point it out and a different language or technique may fix the specific problem you may find.

It is generally bad to optimize because of a guess. If it doesn't work and you know why it doesn't work from actual testing, you'll know that optimization is necessary and have a better idea how to effectively do that. Optimization based on a guess is trying to solve a problem that you don't know exists let alone knowing what that problem is.