The undocumented Android change that led to aCropalypse was reported during beta
Comments
neilv
robviren
I spent 5 years in nuclear power before going into tech and I have sort of joked that the software engineering the goes into AI, IoT, Phones, Cars, etc, is the next Chernobyl waiting to happen. It's all fun and games till these things get connected directly to lives. I actually find the governance of nuclear power a decent model for managing the risk. You mentioned management needs to be held responsible but management will choose short term profit every day until dealing with negative externalities is forced upon them like it is in nuclear.
Imagine if the toxic waste that is the mental impact of social media, or forever insecure devices littering the Internet, or any number of things was obligated to the companies to do their best to fix it. Nuclear reactors put huge money away so that when they are done they are bulldozed back into a green field (Other than what was supposed to be a solved problem of spent fuel).
Anyways, the likelihood of regulation is nil and a global world means those negative externalities will continue on churning till the world agrees we suck at managing risk. Which is also nil. Mental health crisis and insecure tech just isn't as flashy as nuclear in terms of getting regulated in the interest of public health.
raverbashing
I second this
Risk management (especially with the js people) seems to be "lol what's that?"
Also we have the C cowboys pretending they can manage the complexity of their "simple language" in a complex world
lupire
This bug has nothing particular to do with C. It's an API design problem ther goes back to weird POSIX choices.
belter
If the C are cowboys I am afraid to ask how the C++ ones would be called...
int_19h
Well, speaking as a C++ dev, it's quite a hippo rodeo.
to11mtm
> We're going to need software engineers to be accountable for things we've done and signed off on.
Only if management is held equally accountable.
I've worked at places with dedicated security teams, that acted as a 'buffer' between management and engineers. Those places tended to take security seriously.
Places -without- those, I noticed that management has a tendency to sweep security issues 'under the rug' since they impact deadlines.
nuc1e0n
Management earns the big bucks because they are supposed to be accountable. If the expectation is that software engineers should be accountable and not management the money should go to the engineers instead.
izacus
When was the last time you've seen any manager being accountable for any defect in software?
dingusdew
[dead]
FpUser
>"We're going to need software engineers to be accountable for things we've done and signed off on."
And see them evaporate from the companies that would implement something like this.
Personally I would neve take a contract where I could be held legally responsible for bugs.
And ultimately it is the owner who let the product out in a wild without spending a year testing billions of permutations first. Try holding them responsible and see how far can you get.
eviks
What billions of permutations do you need to test when the test had already been done by someone else and brought to your issue tracker on a silver platter?
Eisenstein
'Hard' engineers do this, though. If you want the label you should take the responsibility as well.
cookiengineer
> (Hardware) engineers do this, though.
Broadcom exploits over the years would disagree. So would Samsung's hardware encryption. So would TPM. So would BadUSB. So would everything related to Bluetooth. Et cetera.
No matter what kind of engineering you do, don't try to predict the future. Engineer towards ease of maintenance and ease of update/deployment instead.
FpUser
I do not give a shit about a label and have none in a software field. Just MSc in physics. I design and implement products and I have long list of clients happy with my work. This helps me find new contracts. You can call me a shrub if you wish. I still get paid.
anthk
In Europe calling yourself an enginner without having a proper title can send you to jail, because an actual Engineer has civil duties and accountability on civil causalities.
throw_m239339
[flagged]
hedora
I don’t think that’s true.
None of the engineers I have hired or worked with assume such liability directly.
Usually, their firms limit liability to the price of the contract. I assume insurance does otherwise.
BeFlatXIII
OK, now we're programmer code monkeys (but doing the exact same work as when we pretended to be engineers)
theK
Never heard that term. Who calls themselves a "hard" engineer?
washadjeffmad
No one, but there are PEs.
The frivolity with which "Engineer" titles were co-opted by tech has always been distasteful to me. It felt intended to claim all of the prestige with none of the rigor, accountability, or social responsibility.
salawat
I want a software PE program so bad. I've done my best to hold myself to the same ethical standards, but if there isn't a legal bite behind negligence, you just get shoved aside.
FpUser
Unless you work in regulated sector where lives / critical infra literally depends on your code no employer will pay for "PE level ethical standard" development. Too expensive. So your PE will be of very limited value unless you want to prohibit people without PE making a living writing software in general. But if that is what you want my take is - GFY.
narag
It seems it's a play with soft-ware hard-ware, implying electric and civil engineering.
usefulcat
> And see them evaporate from the companies that would implement something like this.
It might be doable, but would also require (at a minimum) a lot more compensation to make up for the increased liability. Which gets to the heart of the problem, it’s a lot more expensive to produce higher quality software.
wirrbel
Company needs to be liable and introduce necessary steps so that it’s staff operates responsibly.
How quick is a manager at deprioritizing quality to secure a deadline?
rofrol
I cannot comment on threads older than 2 weeks https://news.ycombinator.com/item?id=10199405
So reply to this https://news.ycombinator.com/item?id=35164665
After cloning astrovim as `~/.config/nvim`, do similar with https://github.com/AstroNvim/user_example but with button "Use this template".
Then in `~/.config/nvim/user/plugins/user.lua` add
``` return { { 'Mofiqul/vscode.nvim', priority = 1000, config = function() --vim.cmd.colorscheme 'vscode' vim.o.background = 'light'
local c = require('vscode.colors').get_colors()
require('vscode').setup({
-- Alternatively set style in setup
-- style = 'light'
-- Enable transparent background
transparent = true,
-- Enable italic comment
italic_comments = false,
-- Disable nvim-tree background color
disable_nvimtree_bg = true,
-- Override colors (see ./lua/vscode/colors.lua)
color_overrides = {
vscLineNumber = '#FFFFFF',
},
-- Override highlight groups (see ./lua/vscode/theme.lua)
group_overrides = {
-- this supports the same val table as vim.api.nvim_set_hl
-- use colors from this colorscheme by requiring vscode.colors!
Cursor = { fg=c.vscDarkBlue, bg=c.vscLightGreen, bold=true },
}
})
require('vscode').load()
end,
},and in `~/.config/nvim/lua/user/init.lua`:
``` colorscheme = "vscode", ```
Example here https://github.com/rofrol/AstroNvim--user_example
wirrbel
Thanks
yjftsjthsd-h
> We're going to need software engineers to be accountable for things we've done and signed off on.
Management first. It's not engineers setting incentives.
lupire
At a properly licensed engineering firm, the managers are senior engineers who review and sign off.
WirelessGigabit
Keyword being properly. Too many companies push their engineers into people management.
Which is weird. Your best people should remain engineers. Have them indeed sign off on these things.
chii
> Your best people should remain engineers. Have them indeed sign off on these things.
I agree; the problem with an engineer track is that the salary ceiling is too low, and you have to move to management to get higher salary. but then you end up not using your engineering skills, and waste potential.
What needs to happen is a way for people management to be done without a hierarchical structure. Aka, the senior engineer sits not under their manager, but that the manager is subservient.
Joel spolsky have written about this "problem" already in the last couple of decade: https://www.joelonsoftware.com/2009/03/09/how-to-be-a-progra...
> To make sure that the debate happens respectfully and on a rational basis of facts, it’s absolutely critical that the program managers and developers be peers. If developers report to the program manager, at some point during the debate the program manager is going to get sick of the whole thing and just say, “OK, enough talking, now we do it my way.”
ncphil
The salary range in a professional track does not _have_ to be too low. That's a choice, a value judgment, by management. Some companies do things differently. In at least one company I know, the professional track runs parallel to the management track: engineers can earn as much or more than senior managers, architects as much as directors. It's all up to whether the organization really values the services it's independent contributors provide. Sadly, many companies are not only indifferent, but hostile to the whole concept: playing at cost reduction theater and outsourcing to satisfy the avarice of simpleton investors. Bottom line is that tech pros need to take charge of their own futures and avoid the latter kinds of companies like the plague, when they have a choice. People need to make a habit of asking questions about professional compensation structures during interviews.
tgv
Indeed. I haven't worked at a company that was managed (partially) by engineers. Sure, there were tech leads, but as soon as it came to product and account management and "up", it was business school. Must be some corollary of Peter's Principle.
theK
I don't think pushing a good engineer to also take on management work is such a bad idea. I've actually had quite a lot of success with that system.
The much worse idea is to promote BAD engineers to full time management positions, which seems to be the way most sub par and mediocre companies do it right now.
lupire
The meme is that great engineers make bad people managers. They can still be engineering managers, and have a business/HR people run the org and try to make underling engineers productive, while the lead engineer is responsible for certifying whatever gets engineered.
theK
I don't think this is black or white. I've found out that brilliant engineers actually can be incredible people managers if you help them develop those soft skills. Sure there is the age old adage that "I am a brilliant engineer just pay me more for my engineering work" but this has two aspects. For one you definitely can do it if your org is big and rich enough. But on the other side, software is a product of people interacting. If your engineering importance rises you still will have to deal with lots if people. So are you really staying purely an engineer then?
WirelessGigabit
The problem is when this is the only path to growth. You basically cap out as Senior Engineer.
theK
Is that an argument for the former or the latter?
WirelessGigabit
I think that the choice should be up to the person. But there almost never is a non-managerial path.
ryandrake
This is just passing the buck. If the boss tells you to write credit card stealing software or to DDOS your competitor are you just going to do it? As professionals, we need to push back and refuse when we know our practices are letting critical bugs go to production, just like (I assume) we'd push back if we were asked to build something unethical. And we should be accountable if we don't.
If a civil engineer signs off on a bridge and it collapses during rush hour, he doesn't get to just say, "Whelp, boss rushed me, blame him."
BobbyJo
Your example is so decoupled from reality that it's misleading. Most catastrophic problems in large systems aren't "I made a thing and it's buggy," or "I made a thing that steals SSNs." Most of them are some variation of "It used to work this way, but this upstream dependency changed in a way that we are ok with, and now it works a different way, and we have no visibility into what that may change for downstream dependencies, so hopefully they are aware of the same things we are and act accordingly." Bugs can come from many different people signing off on changes that are often, in isolation, fine.
Imagine if all buildings had to constantly be retrofitted to support a new structures on, below, inside of, around, and next to them, and every retrofit had a different engineer/architect/etc. No civil engineer would ever sign off on anything.
ryandrake
Yours is a good example! I have in a past life actually used the "you wouldn't bolt a new floor onto a building" argument successfully to push against feature creep.
I think even a small amount of accountability would definitely start some much-needed conversations at software companies about things like "do we really need to retrofit this thing onto that working thing?" and "do we fully understand our dependencies and the consequence of merging from them?" and software quality would be better for it. If the consequence of a bug is low, go for it--the accountability should be low, too.
EDIT, just to add another example. I remember when I felt rushed and checked in some client code that did not implement exponential backoff for request failures--it just retried. Of course that code made it to production and our customers started DDOSing our server when a brief outage happened. I felt terrible about it and honestly believe I should have been fired for it. Yes, management rushed me, but they're not the ones that wrote the code. They're not the ones that made an amateur-hour error. It was me.
scns
Interesting. Great that you want to take full responsibility for your work. There is always two sides to a coin though. Strange to me that you scapegoat yourself that hard. Reminds me of the story, where someones error cost the compny millions. He gets called in his superiors office. He says something about resignation. His boss responds: "What? We just invested millions in your education". Every error we still have to make, is a treasure waiting to be dug up IMHO.
mangamadaiyan
> I felt terrible about it and honestly believe I should have been fired for it. > Yes, management rushed me, but they're not the ones that wrote the code
If you felt so strongly about it, you could have quit. Why wait on management to fire you?
It's fairly easy to blame engineers for everything that goes wrong. Engineering doesn't work in a vacuum, the system and processes have to have guardrails in place that prevent lapses in quality, including security, from reaching the customer. When such guardrails have been systematically dismantled in favour of "improving productivity", the management - which removed the guardrails - becomes the culpable entity.
Izkata
> Yours is a good example! I have in a past life actually used the "you wouldn't bolt a new floor onto a building" argument successfully to push against feature creep.
I've seen a few houses built on top of apartment buildings though.
ok_dad
> had to constantly be retrofitted
There’s your problem. You can’t constantly change software contracts without causing bugs. If you’re liable for the code chances are things will slow a bit. Good! We need to develop software slower.
BobbyJo
> You can’t constantly change software contracts without causing bugs
You can't write software without causing bugs, static requirements or not. The software that goes into air/space-crafts is written this way, and it still breaks occasionally. Sometimes bugs are circumvented with hardware because it's cheaper and more reliable than making the software change.
Buildings have fault tolerances. You can use the wrong amount of something, or put something in the wrong place, and have it still work. If buildings had to be designed and built exactly corrects or they fell apart, we wouldn't have buildings.
> We need to develop software slower
To each his own. Most software is fine having bugs. I'd rather have a modern web browser with a few bugs than have an ascii-only command line browser that works flawlessly. The U.S. economy would agree with me there. The software in your pacemaker or car, sure, balance safety and speed to save the most lives/time, but most software operates in domains with far lower stakes.
tialaramex
> If a civil engineer signs off on a bridge and it collapses during rush hour, he doesn't get to just say, "Whelp, boss rushed me, blame him."
He doesn't but this is actually a problem. Ideally, he doesn't get to say this because he chose to rush and if he'd said "No" instead then the boss can't do anything about it. In reality, especially in the US, his boss will just fire him and hire somebody else more co-operative and he knows it.
Without a strong, legally supported and well resourced professional organisation behind people with an obligation like this, they will get exploited and then blamed by people who are insulated from the effects.
Do you see such an organisation for software engineers in the United States of America? Or even most of Europe? Because I do not.
robertlagrant
> Do you see such an organisation for software engineers in the United States of America? Or even most of Europe? Because I do not.
The problem is that medicine and law move orders of magnitude more slowly than software, at the moment. Software is evolving unbelieveably faster, and its problems are far more diverse. As it settles down, if it ever does, we may be able to calcify roles more formally, and then regulate those roles. But at the moment it's the wild west.
ryandrake
I fully agree a strong, organized professional organization for software developers would solve a lot of problems the craft faces. If we had something like the Bar Association or the Medical Board, would we have to keep implementing FizzBuzz and reversing linked lists every interview just to prove a basic level of competence?
ElectricalUnion
Not, but then the "Software Developer Board" would apply those as certification tests instead.
mandevil
But you'd take the exam once rather than having to do it every few years when you want to change jobs.
to11mtm
> If a civil engineer signs off on a bridge and it collapses during rush hour, he doesn't get to just say, "Whelp, boss rushed me, blame him."
I'm willing to bet that the civil engineer in that case has much better whistle-blowing protections, or at the very least, some comfort that if they get fired and someone else replaces them that's willing to do it... that person will be the unhireable one in the future.
59nadir
> This is just passing the buck. If the boss tells you to write credit card stealing software or to DDOS your competitor are you just going to do it?
This is already illegal. Holding the makers of things more accountable would do nothing for you in this example.
potatofrenzy
> This "aCropalypse" event is an example of how making the wrong triage of an issue report can turn out very expensive. All the costs to the world of aCropalypse could've been averted.
Not trying to be facetious, but... what costs? It is unlikely that this cost the company any substantial amount. It probably wasn't even a whole lot of extra work for the PR department, as the bug didn't generate mainstream headlines.
neilv
The unquoted sentence after that was "Some of those costs might eventually come back to the company." Which I distinguish from "costs to the world" in the quoted sentence.
By costs to the world I'm thinking all the things that happen because something a user cropped out was revealed because it wasn't really cropped.
Hypotheticals to illustrate costs to the world:
* A company loses a big business deal because IP/plans leaked to competition in reverted-crop.
* A medical provider and personnel get hit by lawsuit and/or HIPPA for patient data de-anonymized due to reverted-crop of an image.
* Someone gets harassed at their workplace when a reverted-crop adult photo revealed their identity and gets circulated among colleagues.
* Semi-automated extortion rackets, going through troves of images, looking for revertible cropping.
* Other businesses caught in the middle of this have to expend resources to mitigate, or even deal with liability for depending on or assuming correctness of third-party tech behavior, etc.
Maybe luck is with us, and not a single instance of harm to the world actually happens. Or maybe there are many such instances of harm to the world.
Regarding costs to the world eventually coming back to a company -- speaking in general, not of this particular situation -- I think that could come in forms including: brand damage, lost business, lawsuits from those harmed, undesirable turns in legislation, regulatory fines, etc.
tgv
> the bug didn't generate mainstream headlines
So ... yesterday it turned out that Microsoft had the control over a certain Bing's search results wide open to anyone with an Azure account, and that enabled anyone to get access to and control over the Azure services of everyone that used Bing for these specific searches. There were probably other ways too, but that hasn't been disclosed or investigated. And it is --of course!-- unknown if this attack ever happened in real life: there were no logs.
That's quite the problem, wouldn't you say? Well, did you see mainstream articles? There was only an article in The Reg, which attracted a mere 11 replies. It simply doesn't get attention, probably because it was framed as "a misconfiguration", yet the costs could be staggering.
numpad0
IMO this is another anecdotal proof that so called software “engineering” isn’t, and how uncontrolled over-complexity of modern software is going further out of control.
Software is written, like blogs, novels or laws, not engineered. That’s wrong.
KennyBlanken
This wasn't a matter of the ball being dropped, this is a matter of the epidemic of arrogance in Google.
It's going to keep biting them in the ass. The prick who heads up Golang's security has refused to allow people to control what encryption algorithms are enabled to disabled in Golang's TLS 1.3 implementation (he was one of the designers of the TLS 1.3 spec) because, and I wish I were joking - he feels TLS 1.3's implementation is perfect and nobody would have a reason to, and even if they do, why, they can just wait until google releases the patch to fix it, and then deploy that change.
Because of course that's how production environments work....you can just deploy a new version of the language you're using, instead of, you know, a config file change or a few lines of code.
uf00lme
> All the costs to the world of aCropalypse could've been averted. Some of those costs might eventually come back to the company.
This is the bigger problem, there will be very little real cost to the company or engineers involved. My guess is that the cost for engineering a fix and blogging it about it falls into the buffer PR spend or security engineering time. If anything the company will learn from this mistakes for a short time and it will actually save them money in the long run.
The vast majority of software used today has very little legal liability. The industry has perfected the art of legal lip service. There is a massive growth at all costs mentality throughout the industry and engineer’s personal pride only goes so far.
If you want to see real change then you need government and the judicial to enforce real legal liability onto software engineers and companies. That would result in real cyber security rather than current wack-a-mole fines or government’s trying to promote or enforce best practices etc.
wilg
What kind of "accountability" do you mean?
Jtsummers
Presumably similar to the accountability that applies to PEs. A subset of engineers are granted authority to sign off on parts of the system, they're assuming a legal responsibility and the penalties for malpractice can go all the way to imprisonment.
Non-licensed engineer are not liable in the same sense, their employers and PEs are. Same thing could apply in software, but people whine too much and it probably won't happen. Every time the topic comes up, people chime in with "But software is different. It's hard." and expect that lame ass excuse to be sufficiently persuasive. Software is also critical, not all, but a lot. Critical systems warrant additional scrutiny and liability.
wilg
What aspect of this "acropalypse" bug amounts to malpractice?
Jtsummers
An undocumented change to a core system behavior that fundamentally altered how (likely) most applications behaved in an unanticipated (for the users of the system and programmers targeting the system) way. That's malpractice.
wilg
Yeah, I don't buy that. Too much Monday-morning quarterbacking.
I'm not against some well-thought-through standards for certain safety-critical applications, but it's got to be a lot better than "maybe code would be better if we threatened to throw engineers in jail for contributing to complex and subtle interdependent bugs that caused a privacy bug that didn't have a significant real-world impact".
raverbashing
And to be even more honest, why create a similar API with different behavior?
How many use cases prefer not truncating files?
masklinn
The worst part is that the API did originally behave like the one it’d originally emulated, before being modified after years of use.
throw_m239339
> We're going to need software engineers to be accountable for things we've done and signed off on.
You start with management and all the way to the top including the CEO and CTO liable for criminal charges.
And, no we're not going to need any of that.
HeavyFeather
> the issue was deferred without acknowledging the reporter's concise suggestions of how to handle it, engineering-wise
A day in my life at my last job. Basically a regular occurrence that it was “urgent” after a customer report.
codethief
This reminds me of how a recent Android version (12?) suddenly made certain characters (e.g. ?:<>) illegal in file names[0], probably because to increase compatibility with NTFS. Wtf, Google. Nowhere was this breaking change announced.
Not only does my notes app, which has worked flawlessly for years, now crash when I accidentally use one of those characters in a file name / note title. (Unfortunately, the app no longer receives updates.) But suddenly my entire ebook library (which follows the name format "first author & second author: title (year)") no longer synchronizes with my phone (running AOSP / GrapheneOS), either. Fantastic.
[0]: https://android.googlesource.com/platform/packages/providers...
mikepavone
Oh, I was wondering my new phone didn't like some of the music files I was trying to copy over using MTP even though the old one handled them fine. I had assumed the phone manufacturer did something dumb, but apparently it was Google all along.
dpkirchner
Makes you wonder what issue they were mitigating. Passing file names as part of a concatenated string to bash? Ala 'bash -c "ls $filename"'?
kps
Windows.
/**
* Sanitizes given name by mutating the file name to make it valid for a FAT filesystem.
*/
codethief
Ah, good point. So it was not NTFS but FAT compatibility they were aiming for.
justin_oaks
Another case of poorly chosen defaults leading to bugs, leading to security issues.
Also a case of violating the Principle of Least Astonishment [0]. How many people would expect that when you write a file you would also have specify that you want it truncated?
[0] https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
jandrese
It's notable that in practically every other programming language on every platform the default is to truncate the file to 0 bytes when opened in write (not append) mode. This isn't one of those things where there are two schools of thought where people take sides. It's a completely baffling decision to buck all convention vs. common sense.
Someone
> It's notable that in practically every other programming language on every platform the default is to truncate the file to 0 bytes when opened in write (not append) mode
Examples of ‘Open’ calls that require setting a bit in the ‘flags’ argument to truncate an existing file:
- https://learn.microsoft.com/en-us/windows/win32/api/winbase/...
- https://man.freebsd.org/cgi/man.cgi?query=open&sektion=2&n=1
moefh
Wow, that Win32 OpenFile function is truly cursed:
- If you don't specify a directory, Windows will look for the file in a bunch of directories, but the current directory is not the first place searched.
- The lpReOpenBuff is annotated as [out], but is an input when uStyle contains the OF_REOPEN flag.
- The function is named OpenFile, but can be used to delete a file. Even better, the documentation is not clear what to do with the return value on success: is it a valid file handle that must be passed to CloseHandle, or just a dummy non-error file handle?
- You can also use it to check the existence of the file -- the documentation says it will open and then close the file. What's the return value in that case? Is it a stale file handle, a dummy, or do you still have to close it?
Someone
You’re forgetting this one: you can specify a flag that makes the call display a dialog box if a requested file does not exist. Yes, this call to open a file can run an event loop that lets the user create directories, rename files, possibly even format floppy disks, etc.
If you want a prime example of a layer violation, this is it.
Its replacement “CreateFile” (https://learn.microsoft.com/en-us/windows/win32/api/fileapi/...) is a lot saner, but still has this:
“If you rename or delete a file and then restore it shortly afterward, the system searches the cache for file information to restore. Cached information includes its short/long name pair and creation time.”
That’s intended to make it easier for “Save” operations in document-based applications to keep the short name of the file, creation time and file flags around.
okanat
You're comparing system call interfaces to higher level standard API defaults. Not really an apples to apples comparison.
System calls are special low-level interfaces. The requirement for reading their docs carefully and general care is expected at that level.
A higher level API shouldn't contain gotchas like this.
intelVISA
Some say certain agencies wanted it this way.
jmull
Some are yet naive.
“Never attribute to malice that which is adequately explained by stupidity.”
account42
Are you trying to volunteer yourself as an example of naivety.
rozab
This would be the stupidest possible way of conducting mass surveillance
bagels
Who says this?
dylan604
They and them. Don't confuse the pronouns as being more than one person.
sitkack
By Design
Won't Fix Working As Intended
Declaring Bug Bankruptcy
Google doesn't test. Google relies on the public to test its software and even then most bugs don't get addressed. If the bug is a gap in the spec or open to interpretation it won't get fixed. Spec bugs are not allowed.
The reason that Google has such a high hiring bar is that most of its processes run open-loop and it relies on having smart thorough people to prevent the large swath of issues from being created in the first place.
lapcat
> The reason that Google has such a high hiring bar is that most of its processes run open-loop and it relies on having smart thorough people to prevent the large swath of issues from being created in the first place.
But there is a large swath of issues. Just look at Google's public bug trackers.
And Google's atrocious practice of auto-closing old bug reports partially covers up how many issues there really are.
btown
A cautionary tale here: when you are testing, if you mock out even low-level I/O utilities, you’re vulnerable to those utilities changing specification subtly. You can supply-chain attack yourself by simply updating a library.
Full filesystem/service-engaging integration tests are helpful but far from exhaustive - here, one wouldn’t have just needed to read the cropped file using standard display systems and count the pixels, but hash the underlying file in its entirety.
For critical libraries in an I/O pipeline, encourage team members to read CHANGELOGs - both before choosing a library to make sure they’re maintained, and closely when even doing a minor upgrade.
DoctorNick
Change logs wouldn’t have helped here, this behavior change was undocumented.
btown
IMO the Android team should have realized the Android team was not creating internal libraries at a sufficient level of stability and change tracking for the Android team. This quickly gets into cultural priorities that I have very little context on - but I do think that processes failed here.
amethyst
Unrelated: the nostalagia of iliana's favicon is incredible. So much time spent on old modems watching that animation spin while waiting for pages to slowly load in...
neilv
That one was great, and stylish.
The original NCSA Mosaic one with the globe could've been a bit much, but it was appropriate, given the impact of the Web, even for pre-Web Internet natives: "OMG, this is a tingly step forward in distributed global hypertext, it's happening right now -- interacting around the world, and also superpowering the world." With implied altruistic and benevolent goodness.
(The earlier Mcom/Netscape throbbing "N", when I guess they were still rebranding Navigator from Mosaic, looked silly even at the time. Then they did a good one. And there were a bunch of creative alternatives, including some easter egg ones, which JWZ cataloged at one time.)
TazeTSchnitzel
I wonder if some developer of an app working with ZIP files will have noticed this bug and had to update their app. ZIP is an unusually terrible file format because it has a footer rather than a header, so the truncation bug would lead to a ZIP file with two headers!
canucker2016
Some ZIP-file handling apps go straight to the end of the ZIP file and work backwards looking for the central directory.
I believe others will start from the beginning of the file and parse each local directory header and 1) build up their own central directory truth, or 2) stop once they've reached the start of the central directory.
I recall that the footer-only format was to enable streaming out a ZIP file esp. when rewinding to the front of the file/stream was not possible, like sending over a network - at least that's one of the use cases I recall.
ladberg
I believe the footer-only format was more about being able to make modifications by only appending to the zip file and not having to rewrite/shift any data.
canucker2016
...the central directory needs to be shifted by the amount of the new compressed data + local directory header.
But that's a lot less than if the central directory was at the beginning and the code had to grow the central directory and shift ALL the compressed data + local directory headers down.
ladberg
Yeah that might be the ideal way to do it, but technically you could just append the new entries and new central directory without even removing the old one.
I'm not entirely sure what storage system that would make sense on, but presumably there was one back in the day.
canucker2016
3.5" floppies - 800K-1.44MB, small and slow.
Better than cassette tapes though...
cesarb
> 3.5" floppies - 800K-1.44MB, small and slow.
I feel like a lot of people in this thread are missing a crucial part of the historical context:
ZIP files can be split. That is, a single ZIP file can span more than one floppy disk. The entry for each compressed file in the central directory, which is always on the last floppy disk, has a field which says in which disk the compressed file can be found. If you had a multi-disk ZIP file, and wanted to extract a single file from it, you inserted the last disk, pointed the unpacking software at it, and it would tell you which disk(s) to insert (in sequence) to extract that file.
A lot of the strange decisions of the ZIP file format make sense if you think about that use case: you have a directory full of files, which you want to compress into a set of floppy disks; and once you have that set of floppy disks, you want to extract a subset of the compressed files. You also want to be able to extract the undamaged files even when the last floppy disk (with the central directory) is damaged.
layer8
Same for PDF. But at least that’s much less likely to create the issue of invisible non-deleted data as in the PNG case.
jasonhansel
Why would cropping alter the file in-place? That would presumably leave the image corrupted if (say) your phone died during the cropping process; it would seem much better to save the cropped image separately and then overwrite the original file atomically. After all, it's the same amount of data to write either way.
ledauphin
this comment should be higher.
atomic move semantics are the single thing that makes writing basic filesystem interactions plausibly deterministic. anything else is just pretending that errors never happen.
canucker2016
Mentioned in the link.
This seems to be the original PR for the API in question, ParcelFileDescriptor#parseMode():
https://android.googlesource.com/platform/frameworks/base/+/...
+ public static int parseMode(String mode) {
+ final int modeBits;
+ if ("r".equals(mode)) {
+ modeBits = ParcelFileDescriptor.MODE_READ_ONLY;
+ } else if ("w".equals(mode) || "wt".equals(mode)) {
+ modeBits = ParcelFileDescriptor.MODE_WRITE_ONLY
+ | ParcelFileDescriptor.MODE_CREATE
+ | ParcelFileDescriptor.MODE_TRUNCATE;
+ } else if ("wa".equals(mode)) {
...
But we jump ahead a few years and the code is reworked.
from https://android.googlesource.com/platform/frameworks/base/+/...:
public static int parseMode(String mode) {
- final int modeBits;
- if ("r".equals(mode)) {
- modeBits = ParcelFileDescriptor.MODE_READ_ONLY;
- } else if ("w".equals(mode) || "wt".equals(mode)) {
- modeBits = ParcelFileDescriptor.MODE_WRITE_ONLY
- | ParcelFileDescriptor.MODE_CREATE
- | ParcelFileDescriptor.MODE_TRUNCATE;
- } else if ("wa".equals(mode)) {
...
- return modeBits;
+ return FileUtils.translateModePosixToPfd(FileUtils.translateModeStringToPosix(mode));
}
from https://android.googlesource.com/platform/frameworks/base/+/...:
+ public static int translateModeStringToPosix(String mode) {
+ int res = 0;
+ if (mode.startsWith("rw")) {
+ res |= O_RDWR | O_CREAT;
+ } else if (mode.startsWith("w")) {
+ res |= O_WRONLY | O_CREAT;
+ } else if (mode.startsWith("r")) {
+ res |= O_RDONLY;
+ } else {
+ throw new IllegalArgumentException("Bad mode: " + mode);
+ }
+ if (mode.indexOf('t') != -1) {
+ res |= O_TRUNC;
+ }
+ if (mode.indexOf('a') != -1) {
+ res |= O_APPEND;
+ }
+ return res;
+ }
A dev was careless, plain and simple.
Having code in a monorepo doesn't help if you don't check how your code changes affects consumers/clients of the code.
If they couldn't change translateModeStringToPosix() behaviour, then the dev could've amended parseMode() to map "w' to "wt" to keep the old behaviour (ugly and non-orthogonal, yes, but that's the way the API was designed).
canucker2016
Looking at the reworked parseMode API some more and it looks like a total mess.
Original code would only accept one of these modes: "r", "w", "wt", "wa", "rw", "rwt".
Anything else, and you'll get an IllegalArgumentException exception.
which matches the docs:
from https://developer.android.com/reference/android/os/ParcelFil...
Parameters
mode String: The string representation of the file mode. Can be "r", "w", "wt", "wa", "rw" or "rwt".
Throws
IllegalArgumentException if the given string does not match a known file mode.
The modified parseMode code will only throw the IllegalArgumentException if the mode string does NOT start with "rw", "w", or "r".
You can append any other character(s) to the mode string ("a", "t", "x", "y", "z", your fav emoji char), and you won't get an IllegalArgumentException exception now.
I wonder what happens when open() gets a mode flag with both "a" and "t" set?
parseMode API contract thrown out the window...
masklinn
> I wonder what happens when open() gets a mode flag with both "a" and "t" set?
Nothing I expect. O_TRUNC and O_APPEND are not incompatible, O_APPEND is just redundant (because by definition if you truncate the file you're writing at its end).
macOS certainly doesn't seem to care when I do it.
cesarb
> O_APPEND is just redundant (because by definition if you truncate the file you're writing at its end).
No, O_APPEND is not redundant with O_TRUNC. When you use O_APPEND, you also have the extra behavior that every write goes to the end of the file, instead of the current position; this is important when more than one process is writing to the same file.
makomk
The original C/POSIX API this is based on is non-orthogonal because the combinations it doesn't support generally don't make sense. There's usually no point opening files for reading or in append mode if you're truncating the existing contents anyway, and not truncating when opening in write-only mode is almost always wrong because any existing contents that aren't overwritten are almost certainly not relevant to the new contents and shouldn't be there (because the app clearly isn't payint attention to what was already there).
lupire
That's not true because fdopen has different behavior from fopen, so it's a subtle poor choice that goes back to the beginning.
cryptonector
> Having code in a monorepo doesn't help if you don't check how your code changes affects consumers/clients of the code.
It most definitely does help with that because there's fewer places to look. But if you only look at your monorepo then you'll miss a lot of things if the API you're touching is public. In this case there were consumers in the same monorepo were there not?
The issue here is that someone dropped the ball on backwards compatibility. That happens, especially when the people involved are inexperienced.
lupire
The original author made a huge mistake of not loudly commenting the intentionally weird behavior that was mimicking the weird POSIX API, and not having any functional tests that depended on the originally designed behavior.
JoeAltmaier
I have sympathy. It's hard to write fundamental APIs. They need to grow and change like anything else, always at the risk of startling somebody.
Years ago I extended a network file system server to include additional information in an Open response (revision tag to enable distributed caching). Turns out a customer had also extended the server response in their builds, to do something else security-related. Our release startled them, resulting in a lawsuit.
It was all silly and fixed in a patch (a boot config file enable/disable of our new extended info) and it went away. But it underlines how with the best of intentions, still things go wrong.
jimrandomh
It looks like this was reported as a normal bug, and the security implications weren't noticed until later. This is unfortunate, but it's also pretty normal and hard to avoid this sort of thing happnening.
layer8
It was declared to be an intentional change and not a bug. If anyone would have thought through how an application expecting the old semantics will behave under the new semantics, it should have been rather obvious that silently changing the semantics is a very bad idea. It’s absolutely a case of professional negligence.
cvccvroomvroom
Developers at big companies aren't motivated to do everything perfectly. They're graded for big wins, not tech debt, documentation, or correctness.
lupire
> This bug was closed as obsolete a year later.
The infamous "Bug Bankruptcy"
OnlyRepliesToBS
[dead]
Kudos to the engineer who originally not only identified the problem, but also raised the concern about breaking behavior, and about documenting it.
Looks like the ball was first dropped when the issue was deferred without acknowledging the reporter's concise suggestions of how to handle it, engineering-wise. Nor a sign that existing code was reviewed for the same problem.
Then looks like the ball was dropped again, in what I'm guessing might've been: "old version, forget all the old open issues, all the issues that still apply, someone will rediscover the hard way".
This "aCropalypse" event is an example of how making the wrong triage of an issue report can turn out very expensive. All the costs to the world of aCropalypse could've been averted. Some of those costs might eventually come back to the company.
It's easy to guess why the problem happened and the report wasn't handled responsibly. By the time the problem exhibited in a way that couldn't be ignored, the pertinent metrics/KPIs/OKRs about clearing issues, resource allocations, and product shipment schedules were already in the past, bonuses had been paid, promotions for shipping new things earned, etc. And security problems are treated as inevitable, even if there's a constant stream of them, and they're produced faster than they're fixed.
We're going to need software engineers to be accountable for things we've done and signed off on.
(Bonus if accountability changes happen near-term: GPT-powered open source laundering gets a pause, because bridges start falling on everyone foolish enough to sign off on that mangled statistical plagiarism.)