py/objint: Fix int.to_bytes() buffer size checks.

Fixes and improvements to `int.to_bytes()` are:
- No longer overflows if byte size is 0 (closes #13041).
- Raises OverflowError in any case where number won't fit into byte length
  (now matches CPython, previously MicroPython would return a truncated
  bytes object).
- Document that `micropython int.to_bytes()` doesn't implement the optional
  signed kwarg, but will behave as if `signed=True` when the integer is
  negative (this is the current behaviour).  Add tests for this also.

Requires changes for small ints, MPZ large ints, and "long long" large
ints.

Adds a new set of unit tests for ints between 32 and 64 bits to increase
coverage of "long long" large ints, which are otherwise untested.

Tested on unix port (64 bit small ints, MPZ long ints) and Zephyr STM32WB
board (32 bit small ints, long long large ints).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit is contained in:
Angus Gratton
2023-11-29 11:23:16 +11:00
committed by Damien George
parent d933210d96
commit 908ab1ceca
12 changed files with 302 additions and 29 deletions

View File

@@ -57,10 +57,27 @@ mp_obj_t mp_obj_int_from_bytes_impl(bool big_endian, size_t len, const byte *buf
return mp_obj_new_int_from_ll(value);
}
void mp_obj_int_to_bytes_impl(mp_obj_t self_in, bool big_endian, size_t len, byte *buf) {
bool mp_obj_int_to_bytes_impl(mp_obj_t self_in, bool big_endian, size_t len, byte *buf) {
assert(mp_obj_is_exact_type(self_in, &mp_type_int));
mp_obj_int_t *self = self_in;
long long val = self->val;
size_t slen; // Number of bytes to represent val
// This logic has a twin in objint.c
if (val > 0) {
slen = (sizeof(long long) * 8 - mp_clzll(val) + 7) / 8;
} else if (val < -1) {
slen = (sizeof(long long) * 8 - mp_clzll(~val) + 8) / 8;
} else {
// clz of 0 is defined, so 0 and -1 map to 0 and 1
slen = -val;
}
if (slen > len) {
return false; // Would overflow
// TODO: Determine whether to copy and truncate, as some callers probably expect this...?
}
if (big_endian) {
byte *b = buf + len;
while (b > buf) {
@@ -73,6 +90,7 @@ void mp_obj_int_to_bytes_impl(mp_obj_t self_in, bool big_endian, size_t len, byt
val >>= 8;
}
}
return true;
}
int mp_obj_int_sign(mp_obj_t self_in) {