8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name
only though! After fromating it with ext4 it only had 15TB of usuable
space.
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name only though! After fromating it with ext4 it only had 15TB of usuable space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets
you down to 15.5TB, to which you also have to remove the space used by inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by directories and metadata.
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name >> > only though! After fromating it with ext4 it only had 15TB of usuable
space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets
you down to 15.5TB, to which you also have to remove the space used by
inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by
directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name >>> only though! After fromating it with ext4 it only had 15TB of usuable
space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets
you down to 15.5TB, to which you also have to remove the space used by
inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by
directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
TB is about 10% larger.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes.
I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
On 1/7/25 10:44, Dan Purgert wrote:
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name >>> only though! After fromating it with ext4 it only had 15TB of usuable
space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets >> you down to 15.5TB, to which you also have to remove the space used by
inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by
directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
My intuition votes for 1-1000/1024 = 1-125/128 which is approximately 2.35% but it's been wrong before.
On Tue, Jan 07, 2025 at 11:05:01AM -0500, eben@gmx.us wrote:
On 1/7/25 10:44, Dan Purgert wrote:
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name >>>>> only though! After fromating it with ext4 it only had 15TB of usuable >>>>> space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets >>>> you down to 15.5TB, to which you also have to remove the space used by >>>> inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by
directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
My intuition votes for 1-1000/1024 = 1-125/128 which is approximately 2.35% >> but it's been wrong before.
That would be for KB, but Tera is the third power of that. So it's about three times 2.35%, if you throw away the higher order terms (we physicists are cheap, like that ;-)
On Tue, Jan 07, 2025 at 10:44:00AM -0500, Dan Purgert wrote:
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name
only though! After fromating it with ext4 it only had 15TB of usuable space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets you down to 15.5TB, to which you also have to remove the space used by inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
TB is about 10% larger. One of the worst crimes in computer history
was ever talking about storage in powers of 2, I really wish it would
just go away. It has properties that nobody wants and has been the
source of endless confusion, for really no benefits whatsoever.
Merchants insist on decimal only because their cash registers have no
buttons for hex digits.
0xA exp 0xC is 0xE8d4A51000
0x2 exp 0x28 is 0x10000000000
0x10000000000 / 0xE8d4A51000 = ~ 0x1.197D938
So it's 0x19.8 per 0x100 loss for us hard working programmers when the scrooges point to the International System of Units as justification for giving us only a single-digit power of 0xA rather than a double-digit
power of 0x2.
I hope to have made my case sufficiently enough to get programmer's Teras next time i buy a disk.
That would be for KB, but Tera is the third power of that. So it's about three times 2.35%, if you throw away the higher order terms (we physicists are cheap, like that ;-)
I point people to http://www.tarsnap.com/GB-why.html which is where I
was first enlightened.
On Tue, Jan 07, 2025 at 10:44:00AM -0500, Dan Purgert wrote:
On Jan 07, 2025, Stefan Monnier wrote:
8 TB is not that big. I have a external 18 TB drive. It is 18 TB in name >>> > only though! After fromating it with ext4 it only had 15TB of usuable
space.
18TB "on paper" is usually 18 * 1000^4 bytes, so if you convert this
into "computer units" is ~16.37 * 1024^4 bytes. If you then make an
ext4 filesystem on it with the customary 5% reserved for root, that gets >>> you down to 15.5TB, to which you also have to remove the space used by
inodes, so yes, probably about 15TB and of course, once you start
putting actual files ion the drive, additional space will be used by
directories and metadata.
Now now, let's not derail a rant with facts :)
That being said, I thought the variance from TB -> TiB was 10%; or have
I gotten it backwards?
TB is about 10% larger. One of the worst crimes in computer history
was ever talking about storage in powers of 2, I really wish it would
just go away. It has properties that nobody wants and has been the
source of endless confusion, for really no benefits whatsoever.
Mr. Tarsnap forgets something. The reason disks are addressed in powers
of two has to do with mathematics. Every hard and floppy disk out there
has flaws. To get around that, data is divided into sectors, and
checksums calculated. Done right, this allows for error correction for
small flaws. The math works out better if you do it in chunks that are >integer powers of two. So floppy disks have sectors of 256 octets, and
their attendant checksums. Modern hard drives schlep data in chunks of
4096 (2^12) octets. And bytes these days are eight bits.
GB or GiB? I don't care, just be clear which one you are using.
I point people to http://www.tarsnap.com/GB-why.html which is where I
was first enlightened.
On Tue, 07 Jan 2025 11:09:06 -0800
Kushal Kumaran <kushal@locationd.net> wrote:
I point people to http://www.tarsnap.com/GB-why.html which is where IMr. Tarsnap forgets something. The reason disks are addressed in powers
was first enlightened.
of two has to do with mathematics. Every hard and floppy disk out there
has flaws. To get around that, data is divided into sectors, and
checksums calculated. Done right, this allows for error correction for
small flaws. The math works out better if you do it in chunks that are integer powers of two. So floppy disks have sectors of 256 octets, and
their attendant checksums. Modern hard drives schlep data in chunks of
4096 (2^12) octets. And bytes these days are eight bits.
This business of using powers of ten to describe hard drives came from
hard drive marketing weanies. They realized that using powers of ten
made their drives *look* larger to the uninitiated. I worked at Maxtor
about the time this was happening, and that's what the marketing
weanies told me.
The marketing weanies used GB for the powers of two numbers and for the powers of ten numbers, which, it it wasn't fraud, skated damned close. Someone since then has come up with GiB, making it possible to
distinguish between the two.
I don't much care. Americans use both imperial measures (miles) and
metric (35 mm film, liter bottles of pop). The agile ones can use both.
Mass is mass, whether you measure it in grams or pounds, on the Earth
or on Luna. My lathe (Hi, Gene) is calibrated in millimeters: one turn
of the handle moves the table 1 mm, and it is graduated in increments
of .05 mm. I have both inch and metric taps, dies and drill bits. And lettered drill bits. GB or GiB? I don't care, just be clear which one
you are using.
Thus ends the rant.
That would be for KB, but Tera is the third power of that. So it's about three times 2.35%, if you throw away the higher order terms (we physicists are cheap, like that ;-)
I think you meant 4th power, but what's a factor 1024 between friends.
> TB is about 10% larger. One of the worst crimes in computer history
> was ever talking about storage in powers of 2, I really wish it would
> just go away. It has properties that nobody wants and has been the
> source of endless confusion, for really no benefits whatsoever.
This makes no sense. The number has the same value no matter which base is >chosen. These are integers, there is no fractional part, so there is no >"uncertainty" about its value. People who need to work with binary integers >usually choose to calculate with them in base 8 or base 16 then convert them >back to base 2. Which does not require calculation just copying.
For example...let's take the 18000000000000B drive discussed earlier. That's 18TB or 16TiB. Annoying, but ok. Now that's also 18000MB but 16763MiB. And it's 18000000MB or 17166137MiB. So if you have a display in MB and you want to know the value in TB you move the decimal 6 places. But if you move the decimal 6 places to get from MiB to TiB you get...the wrong answer. Does
this actually happen? Yes. All the freaking time. (A classic mashup is 1024k blocks expressed with power of 10 M and G.)
For example...let's take the 18000000000000B drive discussed earlier. That's
18TB or 16TiB. Annoying, but ok. Now that's also 18000MB but 16763MiB. And it's 18000000MB or 17166137MiB. So if you have a display in MB and you want to know the value in TB you move the decimal 6 places. But if you move the decimal 6 places to get from MiB to TiB you get...the wrong answer. Does this actually happen? Yes. All the freaking time. (A classic mashup is 1024k
blocks expressed with power of 10 M and G.)
🙂
A related problem is when writing code which displays such sizes in
a human readable way, for example a "speedometer" displaying the number
of bytes per second with a limited amount of space. A common choice is
to use 3 chars for the number plus a unit, i.e. things like "254 kB/s"
or "1.5 MB/s". Now, if you use the "1024" multiplier, you get a funny
quirk when the current speed is, say 1003 kB/s, because "1003 kB/s" uses
one char too many, yet we haven't reached "1.0 MB/s" either.
For example...let's take the 18000000000000B drive discussed earlier. That's 18TB or 16TiB. Annoying, but ok. Now that's also 18000MB but 16763MiB. And it's 18000000MB or 17166137MiB. So if you have a display in MB and you want to know the value in TB you move the decimal 6 places. But if you move the decimal 6 places to get from MiB to TiB you get...the wrong answer. Does
this actually happen? Yes. All the freaking time. (A classic mashup is 1024k blocks expressed with power of 10 M and G.) Now this next part is important: no normal human working with files and disk space and trying to communicate with other normal humans calculates the values in base 8 or base 16. Communication of numbers between ordinary people generally happens in base 10. SI prefixes are base 10. And when you munge up some stupid base 2 units with what people want and expect to be base 10, mistakes and confusion happen. And the benefit of all that confusion and increased cognative load is: absolutely nothing.
A related problem is when writing code which displays such sizes in
a human readable way, for example a "speedometer" displaying the number
of bytes per second with a limited amount of space. A common choice is
to use 3 chars for the number plus a unit, i.e. things like "254 kB/s"
or "1.5 MB/s". Now, if you use the "1024" multiplier, you get a funny
quirk when the current speed is, say 1003 kB/s, because "1003 kB/s" uses
one char too many, yet we haven't reached "1.0 MB/s" either.
For the people who need exact figures, on the other hand, binary units
are much more convenient, not just to measure the size of memory
modules: alignment requirements, maximum sizes of files and devices,
size of stripes, they are all based on powers of two.
On Tue, Jan 07, 2025 at 02:59:47PM -0700, Charles Curley wrote:
Mr. Tarsnap forgets something. The reason disks are addressed in powers
of two has to do with mathematics. Every hard and floppy disk out there
has flaws. To get around that, data is divided into sectors, and
checksums calculated. Done right, this allows for error correction for >small flaws. The math works out better if you do it in chunks that are >integer powers of two. So floppy disks have sectors of 256 octets, and >their attendant checksums. Modern hard drives schlep data in chunks of
4096 (2^12) octets. And bytes these days are eight bits.
The thing is, nobody cares about all that. It's an implementation
detail that matters not to any normal person. Normal people care about
things like "when I just look at the first couple of numbers of the
size in bytes, is it the same thing as the size in <insert large unit>
or do I need to do a bunch of math to answer a simple question?"
GB or GiB? I don't care, just be clear which one you are using.
Which nobody is. The right answer is to stop using power of two units
because they are pointless.
For example, my computers had 5.12 kB,
65.356 kB, 16.777216 MB, 67.108864 MB, 268.435456 MB, 1.073741824 GB,
and 8.589934592 GB of RAM. Perfectly correct, but I prefer to say
they had 5 kiB, 64 kiB, 16 MiB, 64 MiB, 256 MiB, 1 GiB, and 8 GiB of
RAM.
Block devices like floppy disks, hard disks, SSDs, etc. also have
block sizes which are powers of 2, like 256, 512, and 4096 bytes.
This is not because of checksumming as was suggested in this thread,
but because it makes it easier (e.g. for DMA controllers) to copy
from/to pages of RAM.
Therefore, my floppy disks had exacty 170.75 kiB, 720 kiB, and 1.44
MiB --- or as you would like to put it --- 174.848 kB, 737.280 kB, and >1.47456 MB. So again: Are kiB, MiB, GiB, and TiB really pointless?
This is not really confusing, except for people who are too dumb to >understand units and their conversions.
Granted, some confusion came from using k, M, G for the power-of-2 based units.
Back in the days
when we had only kilobytes this didn't matter too much since 2**10 is
so close to 1000 (only 2.4% more) that it was just practical to use
uppercase K to mean "a little more than k", i.e. 2**10, but still
speak of kilobytes. When we reached sizes of megabytes, we couldn't
use a "larger than M letter", so M was simply used for both, 10**6 and
2**20, but it was usually clear from the context, what was meant.
Baloney [...]
It takes 8 bits to make one byte, should we change that to 10 too....
At least with CHS, you got some of the factorisation done for
you, like 14655/64/32 and 9729/255/63, and you can see they're
not based on binary powers.
On Thu, Jan 9, 2025 at 6:45 PM Michael Stone <mstone@debian.org> wrote:
On Thu, Jan 09, 2025 at 09:47:11PM +0100, Nicolas George wrote:
For the people who need exact figures, on the other hand, binary units >are much more convenient, not just to measure the size of memory
modules: alignment requirements, maximum sizes of files and devices,
size of stripes, they are all based on powers of two.
Baloney. People who need to worry about those things are/should be doing that programatically and absolutely do not "need" GiB for anything at
all, certainly not for display. For everyone else, basically all the
time and in every situation, power of ten units make more sense. The
entire computing world has been saddled with this "but a computer
kilobyte is really" nonsense far too long, and it actively hurts UX for everyone other than the vanishingly small set of people who won't shut
up about how important it is to keep that anachronism.
It takes 8 bits to make one byte, should we change that to 10 too....
On Jan 09, 2025, Timothy M Butterworth wrote:Wider bytes were tried early on, flopped. I was once gifted a thing made
On Thu, Jan 9, 2025 at 6:45 PM Michael Stone <mstone@debian.org> wrote:Please no, 8b10b encoding is hard enough. :)
On Thu, Jan 09, 2025 at 09:47:11PM +0100, Nicolas George wrote:It takes 8 bits to make one byte, should we change that to 10 too....
For the people who need exact figures, on the other hand, binary units >>>> are much more convenient, not just to measure the size of memoryBaloney. People who need to worry about those things are/should be doing >>> that programatically and absolutely do not "need" GiB for anything at
modules: alignment requirements, maximum sizes of files and devices,
size of stripes, they are all based on powers of two.
all, certainly not for display. For everyone else, basically all the
time and in every situation, power of ten units make more sense. The
entire computing world has been saddled with this "but a computer
kilobyte is really" nonsense far too long, and it actively hurts UX for
everyone other than the vanishingly small set of people who won't shut
up about how important it is to keep that anachronism.
Past experience shows that we'll live with this for a while (watch
the US still on their Imperial measures,
On 2025-01-10 14:39, John Hasler wrote:
Tomas writes:wasn't a defect in the Hubble telescope mirror caused by a misunderstanding between US/UK what units they were working in?
Past experience shows that we'll live with this for a while (watch
the US still on their Imperial measures,
Pedanticism: The US is not and never has been on the Imperial system.
We use both SI ("metric") and US Customary (the latter predates
Imperial).
On 1/10/25 15:30, mick.crane wrote:That is not what they told us on this side of the pond, it was an error,
On 2025-01-10 14:39, John Hasler wrote:I'd heard it was a failure to account for the effects of non-gravity on the main mirror. The Mars Climate Observer was doomed by one team using
Tomas writes:wasn't a defect in the Hubble telescope mirror caused by a misunderstanding >> between US/UK what units they were working in?
Past experience shows that we'll live with this for a while (watchPedanticism: The US is not and never has been on the Imperial system.
the US still on their Imperial measures,
We use both SI ("metric") and US Customary (the latter predates
Imperial).
imperial instead of metric, and applying the wrong impulse because of it.
--
Friendship is born at that moment when one person says to another:
"What! You too? I thought I was the only one."
-- C. S. Lewis
.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 546 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 155:10:31 |
| Calls: | 10,383 |
| Files: | 14,054 |
| Messages: | 6,417,848 |