cap max column sizes for mssql and oracle #10297
Conversation
Oracle and MSSQL can return column sizes of upwards of 4GB, which can cause memory allocation errors when trying to allocate that much buffer space. This change keeps the prior fix for Postgres compatibility, but caps large column sizes for Oracle and MSSQL databases.
|
|
CT Test Results 2 files 15 suites 5m 11s ⏱️ Results for commit 2e0d5f3. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
|
what needs to be done to move this forward? I see the failing SBOM task, but I'm not sure there's anything I can do to fix that? We cannot upgrade past OTP 26 on our elixir app without fixing this odbc issue. |
I have a task to review this. I also wanted to give it a good thought and check the documentation regarding those column limits, what does each database doc suggest to do about it. If you override the size, what will happen next, will it trim the data? Why allocating 4 GB is a problem, do you have this much data or does it allocate too much empty memory? Are you using 32-bit OS and Erlang? SBOM is fixed in latest master and maint, so don't worry about it, it will need to be rebased at some point. P.S. There also was another merge request in this line #7000 , merged earlier then reverted as causing too much problems, so i'm asked to act cautiously 😊 |
|
Got it. Thank you for the thoughtful response! I appreciate the thorough consideration. The change in #7000 is what I am attempting to fix. You said that change was reverted? But the change introduced there is still on master. |
I did not check whether it actually was reverted or just blamed and named as a candidate for reversal. Maybe this ticket will become its reversal. Anyway i will be looking at both this and 7000, to get something that works. And also the team was interested in some way to automated test this. |
|
@kvakvs @willus10245 No the change was not reverted, it was a consideration for perhaps putting odbc application in a better state for all backends if no one could come up with a real solution which I think is preferable. And of course it would also be fantastic if @willus10245 with the support of @kvakvs could provide a way testing it maybe with a docker solution, as alas I think our windows set up has become rotten and we no longer have a oracle database either so I believe only our postgreSQL test are actually run. |
5 participants
fixes #9302
Oracle and MSSQL can return column sizes of upwards of 4GB, which can cause memory allocation errors when trying to allocate that much buffer space. This change keeps the prior fix for Postgres compatibility, but caps large column sizes for Oracle and MSSQL databases.