Dear Dhruv
Thanks for your reply, can give some more information about it.

I think you misunderstood, ms is talking about unifying “office document” formats, that is single document for a particular “kind/class” of files, and what you were talking about was having one single format for all possible files. So, IMHO, the two are different.

<snip>
As the author, Doug Henschen, himself states Microsoft wont acheive much by providing Adobe’s PDF format as an output option into the Office 2007 suite. Its doesnt help Microsoft in any way, and Adobe gets to boast a lot.
</snip>

Umm.. if i read that article correctly, then the author says the exact opposite. Lemme quote him :

<quote>
The format is the foundation of Adobe’s “Intelligent Documents” product segment, which this year will account for more than one third ($700 million) of the company’s nearly $2 billion in revenue.
…………
To be sure, giving millions of users the ability to turn Office documents into PDFs will knock a lot of wind out of the sails/sales of Adobe and plenty of other vendors. But it still won’t give Microsoft a universal format and universal content creation tool of its own.
</quote>

I think the above clearly shows (and its logical) that adobe won’t exactly get to ‘boast’ anything. For MS, they would be fulfilling a much desired ‘feature’ in MS Office, so then its not a loss for them, just not a very big win

<snip>
No more XLS and DOC and PPTs. Only PDFs. How does that sound?
</snip>

Huh? Who said no xls/doc/ppts?? The option of a pdf output is just that, an option! It won’t replace any of the existing document formats. For that, you would have either the ODF or microsoft’s own format, but not PDF!

<snip>
It also takes away choice and interoperability from the users. True choice lies in the user being able to use the format he wants to based on his needs.
</snip>

Umm, aren’t you contradicting your “one format fits all” funda that you were talking about in your earlier post?

<snip>
Microsoft was coming out with its own unified document format (called OpenXML) based on a ZIP (as in WinZip) archive of XML files.
</snip>

AFAIK, the ZIP file based format is not something that MS is coming out with, its the same style followed by the older StarOffice document files and the ODF. Try renaming any ODF file to a .zip and you will be able to open it and see the various xml files, objects (eg.images) that it contains.

So we agree that _what_ we want to do should be clear

[Moreover, I don’t think….multimedia files and so on.]
Thats exactly the point. Standardisation.

Standardization of what? of file formats? One kind of standardization is having a single file format for a certain “kind/type” of data. For eg. Ogg for audio, or ODF for office-applications.
The kind of standardization that you are suggesting is.. have one single file format for all possible kinds/types of data! But the question is.. Why? What problem do you solve by doing that? What extra feature or facility becomes available with that ?

Now you agree that having one single application that will handle (read/write/edit) ALL kinds of data has is not the right way to do things. I mean, if you want such a thing, then you can just raise your level of abstraction to look at an OS+apps (or distributions in linux), which you view as a big modular application that can handle simply any kind of data/file that you throw at it.

So, if we have different applications, rather more specialized apps, like an multimedia editor for multimedia data, and spreadsheet or office apps for office like data, then whats the point of having a single file format, cos the multimedia app is not going to be able to allow you to do your spreadsheet stuff, or the office app allow you to do video editing, right?

You could say that you can easily “embed” video content in your spreadsheet, but in that case your spreadsheet app uses some components which themselves understand how to deal with that video content. The spreadsheet app simply talks to component through some defined interface, but doesnt have to worry about details of that format itself.

What has been pointed out again by Radical is the concept of shared metadata for all documents which is implemented using the Baegle..

Actually, i didnt say *shared* metadata, i said single defined query-able _format_ of metadata. In other words, any app should be able to query/get details of metadata of any file that it is interested in a defined standard manner. Say, you could have that in xml. I mentioned beagle as an example of what can be achived today, w/o having to change file formats. Basically, beagle understands tons of file formats, and generates a mini-db of all the metadata of the files that it indexes. Now you could hack/alter it to allow you(any app) query the metadata in a defined manner, say a webservice.

One more thing, you mentioned that your motivation for this is that you want to avoid a company having to pay licensing fees to so many different companies for different s/w (like pdf, xls-etc), but for that i’d suggest look at free(dom)-software, rather than this route.

………………
Now the other stuff about ‘ls’ etc, i think thats a completely different use case, its infact a _use-case_, from a users POV! Its a policy in my opinion, i mean, if you are building your system then you could very well choose to have whatever policy that you want, and that would be the ‘behavior’ of your system. Perfectly legal i think

So, lets not mix this with the single file format stuff.

[Moreover, I don’t think....multimedia files and so on.]
Thats exactly the point. Standardisation. And still the interfaces of each application can be different and unique. You would not be limiting your options since the single document format would ensure compatibility with all current document formats, and at the same time try to retain the distinct advantages of each such type. Anyway, I have been discussing this with Radical as well and there seems to be quite a difference of opinion on this “single handle-all application”. AS even you mention, it wont be possible for a single monolithic application to handle the single document format. Even though a single document format might be a remote possibility, a single application to handle that document will totally be against the whole concept of software. Now, even I agree to that viewpoint.

What has been pointed out again by Radical is the concept of shared metadata for all documents which is implemented using the Baegle (sp?) daemon on Linux. I had heard about it in the good old college days, but had totally forgotten about it.

[Regarding your second statement, .... “list of files” file?]
ls can be manipulated (or some other application can be created) to create a list of files in say XML format?? Or the single document format itself, in case we are ready to accept the “single document format” theory. Again the exact format is still in my / our imagination(s).

[For that last point ... Operating System directly?]
In my comment (the second comment on this page) I did mention the shell and not the kernel itself making changes to the file system. And what I meant by my statement was to ask you if the shell is an application itself considered over the core kernel. Of course it is one interface to the kernel and there could be others as well. Just that I needed an example application that was capable of making changes to the filesystem. I could might as well have mentioned somthing like midnight commander or nautilus.

First, how do you plan such a format that would represent all file types & what about the proprietary file formats? May be, a meta tag/metadata for specifying a specific file type/content? In such a case, we’ll have n number of such tags. Again, I am not saying it’s not impossible, but will that be a good/efficient solution?
Moreover, I don’t think story would end there. All programming languages will have to follow such a format. All compilers/Interpreters/file editors will have to be altered for validation. Also, different file formats have their own advantages/dis-advantages. But if we unify them all, we’ll be limiting our options. Moreover, an editor would then be expected to be able to handle all file types, be it compressed files, binary files, multimedia files and so on.

Regarding your second statement, you gave the example:

"-Create it using the ls command to list all mp3 files.
 -Edit it using Vi....physical files"


So, I’ve taken into account one such “list of files” (using ls); for I couldn’t find any format in the example you mentioned. Isn’t that a list of files?? Whatever, point being how do you differentiate between other files & such a “list of files” file, and allow changes in the filesystem?
Secondly, how’s the file format supposed to into account such changes in the filesystem (be it ODF/XML whatever). Are you planning to manipulate the metadata of other files using that “list of files” file?

For that last point, I didn’t really understand the purpose of statement: "Last I checked...the core kernel??"
Where did the shell come into picture? Are you trying to say, “shell is the ONLY interface to Kernel?”
I don’t think all applications are shell dependant for making OS calls & cannot communicate with the Operating System directly?

Normally, there are multiple applications available that can open/edit/apply changes to a particular document
format. Multiple text editors, Audio Editing software, Image editing software exist.

---

Yes, but what I meant was one document format editable in ALL these applications!!! Say an image editable in text editors, audio editors AND image editors. Single document format is the key word there. ODF comes quite close to accomplish that, but leaves out images, audio and video files.

---

If I understand correctly, you mean to say if there is a text file containing a list of files, say in the following format & then if we rename the file x.mp3 to y.mp3 it should be reflected in the file system as well.

-rw-r—– 1 user group 407 Mar 8 09:41 a.mp3
-rw-r—– 1 user group 406 Mar 8 09:41 b.ogg
-rw-r–r– 1 user group 220 Apr 10 06:16 c.wma
-rw-r–r– 1 user group 660 Apr 10 06:13 d.txt
-rw-rw-r– 1 user group 7020 May 27 00:59 e.pdf
drwxr-xr-x 2 user group 1024 Mar 14 07:20 dir1
drwxr-xr-x 11 user group 1024 May 16 09:47 dir2

Now, what I don’t understand regarding this example is:

1. What if you alter the size of the files in such a text file? What if there are multiple files with same name
available?

---

The listing you have provided is a simple ls format listing, which is no format at all. You are right, such issues will crop up in my “fantasy” document format. But thats where metadata comes into the picture. Again, document structures like XML and ODF will take care of this in the future.

---

2. Why should the OS make change to file/filesystem if some text file is edited in a text editor. If you
say, the editor should make the change in the file system regarding the file name it can be done. Anyway, we
cannot expect the OS to monitor changes in such text files and altering other files/filesystem based on changes
made to such text file. The OS should just update the text file being edited in the file-system.

---

I am not saying the OS should make changes to the file / filesystem. Last I checked, the shell is still considered an application over the core kernel?? In any case, eventually it will be the OS which calls APIs and driver level functions that make any changes to the physical files and allocation tables (or any other hardware resources for that matter).

---

1. “One document format, multiple applications able to open, edit, and apply changes to it. Say a simple XML file, based on ODF format.”

Normally, there are multiple applications available that can open/edit/apply changes to a particular document
format. Multiple text editors, Audio Editing software, Image editing software exist.

2. “-Create it using the ls command to list all mp3 files.

-Edit it using Vi and whatever changes you make are reflected on the file system immediately. Say you changed the entry of X.mp3 in the file using Vi and made it Y.mp3. On its own, the shell recognized these changes, and renames the physical files.

-Open the file in a media player and it reads it as a playlist and starts playing the music files. Make changes to the playlist from the media player and automatically the file is updated. (Maybe) The shell reads these changes from the file and automatically applies them to the file system.”

If I understand correctly, you mean to say if there is a text file containing a list of files, say in the following format & then if we rename the file x.mp3 to y.mp3 it should be reflected in the file system as well.

-rw-r—–   1 user     group          407 Mar  8 09:41 a.mp3
-rw-r—–   1 user     group          406 Mar  8 09:41 b.ogg
-rw-r–r–   1 user     group          220 Apr 10 06:16 c.wma
-rw-r–r–   1 user     group          660 Apr 10 06:13 d.txt
-rw-rw-r–   1 user     group         7020 May 27 00:59 e.pdf
drwxr-xr-x   2 user     group         1024 Mar 14 07:20 dir1
drwxr-xr-x  11 user     group         1024 May 16 09:47 dir2

Now, what I don’t understand regarding this example is:

1. What if you alter the size of the files in such a text file? What if there are multiple files with same name
available?

2. Why should the OS make change to file/filesystem if some text file is edited in a text editor. If you
say, the editor should make the change in the file system regarding the file name it can be done. Anyway, we
cannot expect the OS to monitor changes in such text files and altering other files/filesystem based on changes
made to such text file. The OS should just update the text file being edited in the file-system.

One document format, multiple applications able to open, edit, and apply changes to it. Say a simple XML file, based on ODF format.

-Create it using the ls command to list all mp3 files.
-Edit it using Vi and whatever changes you make are reflected on the file system immediately. Say you changed the entry of X.mp3 in the file using Vi and made it Y.mp3. On its own, the shell recognized these changes, and renames the physical files.
-Open the file in a media player and it reads it as a playlist and starts playing the music files. Make changes to the playlist from the media player and automatically the file is updated. (Maybe) The shell reads these changes from the file and automatically applies them to the file system.

Maybe I am asking for too much, but that would be total application level interoperability wouldnt it?

Still unclear? Your comments?

select DISTINCT Field3 from TABLEA t2 where Field3 is not null and t2.Field1=t1.Field1

(syntax could be wrong, but you get the idea)

Oh, and dont worry about meddling me up.. as i have almost no fundaes about this

1. A distinct would still return 2 rows in the given situation. One containing the actual value and the other containing a NULL value. So the functionality required is not obtained.

2. In our situation, this point is not relevant. But DISTINCT will always return distinct rows. That means that it will return distinct values of the entire set of items in the SELECT. To elaborate further on this point, take the case of our TABLEA itself. If we do:

select DISTINCT * from TableA

We will get all rows of the table as the output since the combined set of all fields (including ID) is distinct.

On the other hand if we do:

select DISTINCT Field1, Field3 from TableA

the output would contain only 2 rows, one for the value NULL of Field3 and the other containing the value “546723″ for Field3.

Hopefully I havent meddled you up even more!!!

Quest..err..Query : Why use ‘top n rows’ or the variants instead of a ‘distinct’ on field3 in the inner query?
One reason that i can think of is, that with distinct the server will go through all rows, whereas for a ‘top n’ stuff, it will/should stop at the nth row.. in other words, a distinct would be costlier.. hmm