Ndtyjky

I am struggling with converting a Node application to Ruby. I have a Buffer of integers that I need to encode as an ASCII string.

In Node this is done like this:

const a = Buffer([53, 127, 241, 120, 57, 136, 112, 210, 162, 200, 111, 132, 46, 146, 210, 62, 133, 88, 80, 97, 58, 139, 234, 252, 246, 19, 191, 84, 30, 126, 248, 76])

const b = a.toString('hex')

// b = "357ff178398870d2a2c86f842e92d23e855850613a8beafcf613bf541e7ef84c"

const c = a.toString('ascii') 

// c = '5qx9bpR"Hou0004.u0012R>u0005XPa:u000bj|vu0013?Tu001e~xL'

I want to get the same output in Ruby but I don't know how to convert a to c. I used b to validate that a is parsed the same in Ruby and Node and it looks like it's working.

a = [53, 127, 241, 120, 57, 136, 112, 210, 162, 200, 111, 132, 46, 146, 210, 62, 133, 88, 80, 97, 58, 139, 234, 252, 246, 19, 191, 84, 30, 126, 248, 76].pack('C*')

b = a.unpack('H*') 

# ["357ff178398870d2a2c86f842e92d23e855850613a8beafcf613bf541e7ef84c"]

# c = ???

I have tried serveral things, virtually all of the unpack options, and I also tried using the encode function but I lack the understanding of what the problem is here.

Okay well I am not that familiar with Node.js but you can get fairly close with some basic understandings:

Node states:

'ascii' - For 7-bit ASCII data only. This encoding is fast and will strip the high bit if set.

Update After rereading the nod.js description I think it just means it will drop 127 and only focus on the first 7 bits so this can be simplified to:

def node_js_ascii(bytes) 

  bytes.map {|b| b % 128 }

    .reject(&127.method(:==))

    .pack('C*')

    .encode(Encoding::UTF_8)

end

node_js_ascii(a)

#=>  #=> "5qx9bpR"Hou0004.u0012R>u0005XPa:vj|vu0013?Tu001E~xL"

Now the only differences are that node.js uses "u000b" to represent a vertical tab and ruby uses "v" and that ruby uses uppercase characters for unicode rather than lowercase ("u001E" vs "u001e") (you could handle this if you so chose)

Please note This form of encoding is not reversible due to the fact that you have characters that are greater than 8 bits in your byte array.

TL;DR (previous explanation and solution only works up to 8 bits)

Okay so we know the max supported decimal is 127 ("1111111".to_i(2)) and that node will strip the high bit if set meaning [I am assuming] 241 (an 8 bit number will become 113 if we strip the high bit)

With that understanding we can use:

a = [53, 127, 241, 120, 57, 136, 112, 210, 162, 200, 111, 132, 46, 146, 210, 62, 133, 88, 80, 97, 58, 139, 234, 252, 246, 19, 191, 84, 30, 126, 248, 76].map do |b| 

    b < 128 ? b : b - 128

end.pack('C*')

#=> "5x7Fqx9bpR"Hox04.x12R>x05XPa:vj|vx13?Tx1E~xL"

Then we can encode that as UTF-8 like so:

a.encode(Encoding::UTF_8)

#=> "5u007Fqx9bpR"Hou0004.u0012R>u0005XPa:vj|vu0013?Tu001E~xL"

but there is still is still an issue here.

It seems Node.js also ignores the Delete (127) when it converts to 'ascii' (I mean the high bit is set but if we strip it then it is 63 ("?") which doesn't match the output) so we can fix that too

 a = [53, 127, 241, 120, 57, 136, 112, 210, 162, 200, 111, 132, 46, 146, 210, 62, 133, 88, 80, 97, 58, 139, 234, 252, 246, 19, 191, 84, 30, 126, 248, 76].map do |b| 

    b < 127 ? b : b - 128

end.pack('C*')

#=> "5xFFqx9bpR"Hox04.x12R>x05XPa:vj|vx13?Tx1E~xL"

a.encode(Encoding::UTF_8, undef: :replace, replace: '')

#=> "5qx9bpR"Hou0004.u0012R>u0005XPa:vj|vu0013?Tu001E~xL"

Now since 127 - 128 = -1 (negative signed bit) becomes "xFF" an undefined character in UTF-8 so we add undef: :replace what to do when the character is undefined use replace and we add replace: '' to replace with nothing.