vanowm Posted October 2, 2023 Posted October 2, 2023 (edited) Hello. I'm trying store real number via setenv For that I'm using something like: (setenv "myvar" (rtos 0.5)) which stores string: "1/2" But when I try convert it back to real (atof (getenv "myvar")) [ I get a wrong real number: 1.0 What am I don't wrong here? and how to fix this? Edited October 2, 2023 by vanowm Quote
Tsuky Posted October 2, 2023 Posted October 2, 2023 Force for exemple your storage with (setenv "myvar" (rtos 0.5 2 1)) 1 Quote
Lee Mac Posted October 2, 2023 Posted October 2, 2023 When storing doubles as strings, I like to use a function such as this - ;; Number to String - Lee Mac ;; Converts a supplied numerical argument to a string (defun LM:num->str ( num / dim rtn ) (if (equal num (atoi (rtos num 2 0)) 1e-8) (rtos num 2 0) (progn (setq dim (getvar 'dimzin)) (setvar 'dimzin 8) (setq rtn (rtos num 2 15)) (setvar 'dimzin dim) rtn ) ) ) This ensures that the supplied number is stored to the maximum precision afforded by the double precision floating point format, but in the shortest string necessary to represent the number to the maximum available precision (e.g. integers are stored as integers, doubles have all trailing zeroes removed). 1 1 Quote
Lee Mac Posted October 2, 2023 Posted October 2, 2023 For your example, this would be called thus: (setenv "myvar" (LM:num->str 0.5)) Quote
dan20047 Posted October 2, 2023 Posted October 2, 2023 Lee, Should the 1e-8 fuzz match the precision that you could store floating point values? (max 1e-15?) I suppose it varies depending on how many digits are before the decimal. : (LM:num->str 2.0000000001) "2" : (LM:num->str 2.1000000001) "2.1000000001" : (LM:num->str 2.1000000001123423143) "2.100000000112342" Quote
Lee Mac Posted October 2, 2023 Posted October 2, 2023 (edited) 29 minutes ago, dan20047 said: Should the 1e-8 fuzz match the precision that you could store floating point values? (max 1e-15?) I suppose it varies depending on how many digits are before the decimal. : (LM:num->str 2.0000000001) "2" : (LM:num->str 2.1000000001) "2.1000000001" : (LM:num->str 2.1000000001123423143) "2.100000000112342" The issue with using a fuzz value of 1e-15 is that you may as well not use any tolerance at all since the 1e-15 is already at the limit of precision; furthermore, some numbers cannot be represented exactly in binary (just as others cannot be represented exactly in decimal), and so, following arithmetic operations, infinitesimal errors accumulate at the limit of precision. As such, when the value is to within 1e-8 of an integer, I opted to represent the number as an integer (else one might as well use (rtos <number> 2 15) for everything). If anything, I'd be more inclined to change my (rtos num 2 15) to (rtos num 2 8). Edited October 2, 2023 by Lee Mac Quote
dan20047 Posted October 3, 2023 Posted October 3, 2023 Just for contemplation, perhaps Gilles' routine to calculate fuzz based on point position within the significant digits? https://www.theswamp.org/index.php?PHPSESSID=bt4k9f58n8aemb57ttdik1vvv3&topic=41698.msg469494#msg469494 Thanks as ever for response & posting your code Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.