Previously, I brought up a hypothetical API discussion to see what different people thought about the idea. The basic premise of the discussion was: what do you do when you've published an API to interact with something, and then the rules for that interaction change?
Now I want to make that hypothetical discussion more concrete with a real world issue that's interesting to me. But, I want to preface this with: this post has no bearing on reality. Anything we discuss here is purely academic. I'm in no way saying that REAL Software is going to do something, change anything, drink anything, purchase anything, deploy anything or take over a third world country. Basically: I don't want to hear any "Aaron said" crap at my day job. Got it? ;-)
The hypothetical I discussed before is a generalization of the problems faced by FolderItem.IsWriteable on Windows Vista. Vista introduced this concept called folder virtualization as a mechanism to support older application which did bad things. These older applications made the totally incorrect assumption that the user always ran as admin, and the user could always write to some directories (or registry keys, etc). So, as a back compat measure, the OS virtualizes access to these resources. When a poorly-written application tries to make a file under the Program Files directory, the OS doesn't actually modify the Program Files directory -- instead it modifies the Users\user name\AppData\Local\VirtualStore\Program Files directory. It virtualizes the access, basically.
Before I move on, I'd like to point out one fact (not opinion). These applications were already broken. Any application which assumes (1) the user is an admin, or (2) it can write a file anywhere, is an application that has a bug in it. These are totally incorrect assumptions. So folder virtualization isn't a feature -- it's a workaround for buggy applications.
So the question now becomes: is Program Files writeable or not?
From one perspective, it is. When I say "make a new file here and write to it", a file is created (somewhere), and data is written to it. However, that perspective ignores the "here" problem: the file isn't created where you asked it to be created. It's been created somewhere else entirely! So which is it? Is the FolderItem writeable or not?
My personal perspective on this is that IsWriteable should return whether that location is writeable or not, and a virtualized folder is *not* writeable unless you have the proper access rights to it. If you are not an elevated user, then Program Files is *not* writeable.
My reasoning for taking this stance is fairly simple: if you didn't care about the answer, you wouldn't ask the question in the first place. If you care enough to ask, you should be given the correct answer. The question IsWriteable is asking isn't "will the write operation succeed." The question it is asking is "is this file/folder something I have permission to write to." If you wanted the answer to the previous question, the procedure is quite simple: try to write to the file!
This means that (from my perspective) REALbasic's FolderItem.IsWriteable is returning incorrect information. It's checking the answer to the "will this write work" question on Windows. This has some very interesting ramifications. For instance, it means that you cannot detect folder virtualization with that API, so applications trying to be correct will actually end up being incorrect instead. When the helpful application wants to list only writeable directories, Program Files will show up, which is not what the OS wants (remember: it's just trying to shim up bad applications).
So, with this concrete example of what I was previously talking about, how have your opinions changed? What do you think the proper behavior of FolderItem.IsWriteable should be?
Can you extend the API with an IsVirtualItem so a person can test "IsWriteable" and get the result the OS intends (for backwards compatibility reasons) and add a "IsVirtualized" test so a person can tell if they are going to write to a virtualized item ?
That way nothing old breaks (despite the fact that maybe it should) and correct code can test and "do the right thing" assuming a that single API addition is enough to do the right thing.
Maybe it takes a couple additions to the API to achieve the goal of making it possible to write correct behaving applications.
I agree with you that you should enforce good programming practices. But not everyone would like that.
There are good reasons to check to see if it is writable first, and it might not have to do with permissions. If the user accidentally chose to save to a CD-ROM, then you can't write to it and thus IsWritable would be false. You wouldn't want to try to write out, especially if it is a long task. You'd want to check first. Of course you'd want to check for permission reasons too, but checking ability is another function of IsWritable.
I think extending the API like norman said would be the best option as it doesn't break code, and contentious users would start checking that.
Sure, we could do anything we want, technically. ;-) Here's the way to look at it (for those of you who want both answers):
If you want to answer the question "do I have permission to write here?", you cannot get it answered currently. This is a bug -- you should be able to ask that question and get a straight answer. The way to ask that question is via IsWriteable.
If you to answer the question "will writing here succeed?", you can get this answered currently by actually attempting to open a file for writing.
(This is the proposed idea:)
// Do I have permission?
if f.IsWriteable then
// Will writing here work?
bs = f.OpenAsBinaryFile( ... )
if bs > nil then
// Write to it!
end if
In the end, I think that no one *really* cares whether a folder is virtualized or not. They're interested in two pieces of information: access check and read/writing files. Access check is for things like listing files and folders for a user interface. Read/Writing files is for reading or writing files. If you have code that's doing both, then you're wasting cycles (there's no need to perform an access check before trying to write to a file since the open will fail and you'll get a nil object back). That's not to say people aren't doing it currently -- but with the proposed solution, those people's applications will now *work properly!*
The code above seems reasonable.. But on Vista, would the FolderItem.AbsolutePath return the virtual path or the one you're asking for? If I could do a conditional to see if the path we're asking for is the one we got, that would be fine. Else, I'd have a heck of a time figuring out where to put the file ;-)
@Christopher -- AbsolutePath points to you the path where the FolderItem is pointing to, so it won't alert you of the virtualization at all. If you try this code on Vista, what you end up with is a FolderItem.AbsolutePath pointing to Program Files\Test.txt, and an actual file in the VirtualStore directory.
dim f as FolderItem = SpecialFolder.Applications.Child( "Test.txt" )
dim bs as BinaryStream = f.CreateBinaryFile( "" )
Break
What's even more fun is that under some circumstances, you will also get a great error message when your application closes asking you if the application worked properly or not, and prompting you to restart the application under elevated privileges. I've yet to track down the circumstances under which this dialog is presented to you, but I've seen it several times from user applications which invariably are using folder virtualization without knowing it.
Now that there is a more concrete example...
You could compliment IsWritable with WritableLevel, which could return constants describing Full, Virtual, and None.
The above could work for all platforms... virtual would just never come up for OS X & Linux.
@Christian -- that would break existing code though since IsWriteable currently returns a Boolean.
Actually people probably have to care if a file is written to a virtualized folder :(
If Iwrite a file to a virtual location I need to know that so I ca nread the darned thing back in from the real location won't I ?
Or is the read also virtualized ?
If not then I can write a file that ends up not being where I thought it was when I go to read it.
@Norman -- Well, that brings up an interesting point to the way virtualization works. The user *has* read permission to the directory, but the OS will also check the virtualized folder as well. So in the case of reading it in, you can't run into an issue. If the file is virtualized, it will still be found regardless of which location it resides in.
Well, I've been thinking about it all day long.
If I understand Aarons stance correctly, I agree with it.
If Vista wants to give you read access to the file you're requesting, fine. If Vista doesn't want to give you write access to that same file (in the same path), it should fail in RB. It's up to the developer to ask if that exact file is writable and if not, choose a more appropriate path.
Ah ... wasnt sure if that was the case or not.
So, if it gets written to the virtual folder and read from the virtual folder and this all happens invisibly I'm left wondering where's the issue ?
IsWriteable will say yes, the file gets written to the virtual folder and when I grab a folder item to it it gets read.
No ?
Or did I miss something along the way ?
@Norman -- you missed something, but it was pretty subtle and easy to miss. The whole point is that the Program Files directory *isn't* writeable, but IsWriteable is telling you that it is. What is writeable is the virtual directory, for now.
In OS X/*nix terms, imagine you have a directory whose permissions are 755, but someone (not the owner) checked IsWriteable on it, and it returned true instead of false.
Okay.. Now *I* must be missing something now..
If you're reading from c:\program files\foo\bar.txt and it's writing to somewhere else, you're going to have problems when you go to open c:\program files\foo\bar.txt and no changes have been made to the file! Right?
If it's not writable then it should fail. Create new method(s) for virtualization. Sure in the short term it will break code that as you already noted contained a bug. Aw Shucks people have to debug their own app... Waaaaaa!
No seriously in the long term it's better to have an accurate API that people understand and doesn't have some legacy workaround that later seems nonsensical.
Keep it clean!
~joe
@Christopher -- correct, if the file exists in both locations. Virtual folders are a shim to help poorly written applications along, they're not meant to be a 100% solution. Just good enough for now.
Chris ... that's what I figured to but Aaron corrected that.
Vista looks in both the location yo uasked for AND the virtualized one so the read will work.
But there are other issues.
If the app writes to it's program directory then expects to be able to read that back in when another uesr runs the app tha twill not work becuaes the virtualized write is in the USERS virtual folder, not a shared one where all users can read from.
The issue here is really programs that do the wrong thing and try to save information where they should not.
Telling the truth is always the best policy. I understand the Vista workaround (or hack) as Vista has to work with already compiled software. Even if they were buggy, they used to work as intended and if Vista didn't allow for that then Vista would be seen as breaking existing applications (bad news for Microsoft). But an API for developers is entirely different. In this case, developers are actually creating buggy apps, and the API is enabling that destructive behavior. Obviously, apps written and compiled with the old API should (and would) continue to work using the OS workaround, but anything compiled with the new API would fail. Seems fair.
Of course, I'm sure plenty of people will bitch about the new "bug" the API introduced.
Brady, I agree completely.
@Aaron
You seem to be overly obsessed about breaking old code. Look at the pain that has caused Microsoft over the years. Break the dang code, man! :)
Christian: +1
Breaking old code has more ramifications when you're API's exist in some external component, eg. the VB runtime DLL, DotNet runtime or platform, eg OS * version *.
The RB compiler is not a component and is free from this platform chase. (I'll avoid the esoterics and sematicism of a platform later) If customers choose to build on a newer version of a compiler, they should expect that some of their older targetted code will need to be updated to work on the newer version, and backward compatibility may not be guaranteed. If folks need the old functionality in a newer version of the compiler, then good deprecation guidelines and communication apply.
Back to your Your case study, which is interesting as it relates to the OS, which is basically a monster component called a platform.
When the platform changes, your API to that platform may need to change when your product has a foundational feature claiming cross-platform development ease. If one function call with the same parameters throws different errors and exceptions, or completely reroutes the functionality, first DOCUMENT IT. If you're writing a confusing document with too many rules, then write specific API's. This is equally true to underlying platform components. http://realsoftware.com/feedback/sndjoakz
Hi Aaron,
I didn´t really change my mind, but want to add something: Don´t just imagine the Perspective of the user, but also keep in mind, where the API is used. Think about writing an encapsulation for that API for the use in a high-level-Code-layer, or in a low-level-code-layer. It might be that the Layer expects completely different answers, because the question he has for the API come from completely different contexts.
In most of my codes, for instance, a high-level-layer never would ask something that has to do with files, because files just don´t have representation in high-levels (not in my codes).
that meands, the high-level-code wants to know something like: "Can my user fill the Washing-Machine?" (I try to find something apart from files)
In a low level-layer the answering of this question might have to do with the question, if a file is writeable or not.
Or it could depend on a network-connection or a database or a proxy, whatever.
Yours,
Jens