Umm mind explaining that use case of mp3 files ? i didnt get it..

By Ankit on 06.13.06 4:55 pm

Yeah, even I thought it wasnt quite unclear. Ok…let me try explaining it again.

One document format, multiple applications able to open, edit, and apply changes to it. Say a simple XML file, based on ODF format.

-Create it using the ls command to list all mp3 files.
-Edit it using Vi and whatever changes you make are reflected on the file system immediately. Say you changed the entry of X.mp3 in the file using Vi and made it Y.mp3. On its own, the shell recognized these changes, and renames the physical files.
-Open the file in a media player and it reads it as a playlist and starts playing the music files. Make changes to the playlist from the media player and automatically the file is updated. (Maybe) The shell reads these changes from the file and automatically applies them to the file system.

Maybe I am asking for too much, but that would be total application level interoperability wouldnt it?

Still unclear? Your comments?

By Dhruv on 06.13.06 6:45 pm

I did not understand quite a few things.

1. “One document format, multiple applications able to open, edit, and apply changes to it. Say a simple XML file, based on ODF format.”

Normally, there are multiple applications available that can open/edit/apply changes to a particular document
format. Multiple text editors, Audio Editing software, Image editing software exist.

2. “-Create it using the ls command to list all mp3 files.

-Edit it using Vi and whatever changes you make are reflected on the file system immediately. Say you changed the entry of X.mp3 in the file using Vi and made it Y.mp3. On its own, the shell recognized these changes, and renames the physical files.

-Open the file in a media player and it reads it as a playlist and starts playing the music files. Make changes to the playlist from the media player and automatically the file is updated. (Maybe) The shell reads these changes from the file and automatically applies them to the file system.”

If I understand correctly, you mean to say if there is a text file containing a list of files, say in the following format & then if we rename the file x.mp3 to y.mp3 it should be reflected in the file system as well.

-rw-r—–   1 user     group          407 Mar  8 09:41 a.mp3
-rw-r—–   1 user     group          406 Mar  8 09:41 b.ogg
-rw-r–r–   1 user     group          220 Apr 10 06:16 c.wma
-rw-r–r–   1 user     group          660 Apr 10 06:13 d.txt
-rw-rw-r–   1 user     group         7020 May 27 00:59 e.pdf
drwxr-xr-x   2 user     group         1024 Mar 14 07:20 dir1
drwxr-xr-x  11 user     group         1024 May 16 09:47 dir2

Now, what I don’t understand regarding this example is:

1. What if you alter the size of the files in such a text file? What if there are multiple files with same name
available?

2. Why should the OS make change to file/filesystem if some text file is edited in a text editor. If you
say, the editor should make the change in the file system regarding the file name it can be done. Anyway, we
cannot expect the OS to monitor changes in such text files and altering other files/filesystem based on changes
made to such text file. The OS should just update the text file being edited in the file-system.

By Ankur Jain on 06.15.06 1:52 pm

My comments and thoughts are placed inside hypenated areas. For your consideration and further comments.

Normally, there are multiple applications available that can open/edit/apply changes to a particular document
format. Multiple text editors, Audio Editing software, Image editing software exist.

---

Yes, but what I meant was one document format editable in ALL these applications!!! Say an image editable in text editors, audio editors AND image editors. Single document format is the key word there. ODF comes quite close to accomplish that, but leaves out images, audio and video files.

---

If I understand correctly, you mean to say if there is a text file containing a list of files, say in the following format & then if we rename the file x.mp3 to y.mp3 it should be reflected in the file system as well.

-rw-r—– 1 user group 407 Mar 8 09:41 a.mp3
-rw-r—– 1 user group 406 Mar 8 09:41 b.ogg
-rw-r–r– 1 user group 220 Apr 10 06:16 c.wma
-rw-r–r– 1 user group 660 Apr 10 06:13 d.txt
-rw-rw-r– 1 user group 7020 May 27 00:59 e.pdf
drwxr-xr-x 2 user group 1024 Mar 14 07:20 dir1
drwxr-xr-x 11 user group 1024 May 16 09:47 dir2

Now, what I don’t understand regarding this example is:

1. What if you alter the size of the files in such a text file? What if there are multiple files with same name
available?

---

The listing you have provided is a simple ls format listing, which is no format at all. You are right, such issues will crop up in my “fantasy” document format. But thats where metadata comes into the picture. Again, document structures like XML and ODF will take care of this in the future.

---

2. Why should the OS make change to file/filesystem if some text file is edited in a text editor. If you
say, the editor should make the change in the file system regarding the file name it can be done. Anyway, we
cannot expect the OS to monitor changes in such text files and altering other files/filesystem based on changes
made to such text file. The OS should just update the text file being edited in the file-system.

---

I am not saying the OS should make changes to the file / filesystem. Last I checked, the shell is still considered an application over the core kernel?? In any case, eventually it will be the OS which calls APIs and driver level functions that make any changes to the physical files and allocation tables (or any other hardware resources for that matter).

---

By Dhruv on 06.19.06 11:59 am

You are proposing a Single Document Format, which I’d like to call as Single File Format, that may represent all types of files. And, we expect such a format to be editable by all types of editors. Well, “all types” scenario wouldn’t exist in such a case, for they’ll all edit one single file format.
Of course, that is not impossible; but that leads to a different set of questions.

First, how do you plan such a format that would represent all file types & what about the proprietary file formats? May be, a meta tag/metadata for specifying a specific file type/content? In such a case, we’ll have n number of such tags. Again, I am not saying it’s not impossible, but will that be a good/efficient solution?
Moreover, I don’t think story would end there. All programming languages will have to follow such a format. All compilers/Interpreters/file editors will have to be altered for validation. Also, different file formats have their own advantages/dis-advantages. But if we unify them all, we’ll be limiting our options. Moreover, an editor would then be expected to be able to handle all file types, be it compressed files, binary files, multimedia files and so on.

Regarding your second statement, you gave the example:

"-Create it using the ls command to list all mp3 files.
 -Edit it using Vi....physical files"


So, I’ve taken into account one such “list of files” (using ls); for I couldn’t find any format in the example you mentioned. Isn’t that a list of files?? Whatever, point being how do you differentiate between other files & such a “list of files” file, and allow changes in the filesystem?
Secondly, how’s the file format supposed to into account such changes in the filesystem (be it ODF/XML whatever). Are you planning to manipulate the metadata of other files using that “list of files” file?

For that last point, I didn’t really understand the purpose of statement: "Last I checked...the core kernel??"
Where did the shell come into picture? Are you trying to say, “shell is the ONLY interface to Kernel?”
I don’t think all applications are shell dependant for making OS calls & cannot communicate with the Operating System directly?

By Ankur on 06.19.06 8:34 pm

[You are proposing .... different set of questions. ]
Before I continue this discussion, let me assure you I have no idea how I plan to implement such a format. The whole point of the post was to give an idea, an alternative, however stupid it may sound. The implementation level details are the least of my concerns at the moment. Besides, if what we want to do is clear, then doing it is hardly a problem.

[Moreover, I don’t think....multimedia files and so on.]
Thats exactly the point. Standardisation. And still the interfaces of each application can be different and unique. You would not be limiting your options since the single document format would ensure compatibility with all current document formats, and at the same time try to retain the distinct advantages of each such type. Anyway, I have been discussing this with Radical as well and there seems to be quite a difference of opinion on this “single handle-all application”. AS even you mention, it wont be possible for a single monolithic application to handle the single document format. Even though a single document format might be a remote possibility, a single application to handle that document will totally be against the whole concept of software. Now, even I agree to that viewpoint.

What has been pointed out again by Radical is the concept of shared metadata for all documents which is implemented using the Baegle (sp?) daemon on Linux. I had heard about it in the good old college days, but had totally forgotten about it.

[Regarding your second statement, .... “list of files” file?]
ls can be manipulated (or some other application can be created) to create a list of files in say XML format?? Or the single document format itself, in case we are ready to accept the “single document format” theory. Again the exact format is still in my / our imagination(s).

[For that last point ... Operating System directly?]
In my comment (the second comment on this page) I did mention the shell and not the kernel itself making changes to the file system. And what I meant by my statement was to ask you if the shell is an application itself considered over the core kernel. Of course it is one interface to the kernel and there could be others as well. Just that I needed an example application that was capable of making changes to the filesystem. I could might as well have mentioned somthing like midnight commander or nautilus.

By Dhruv on 06.22.06 12:55 pm

[...] Read it here [...]

By Wired News » Single File Format on 06.22.06 2:50 pm

..Besides, if what we want to do is clear, then doing it is hardly a problem.

So we agree that _what_ we want to do should be clear

[Moreover, I don’t think….multimedia files and so on.]
Thats exactly the point. Standardisation.

Standardization of what? of file formats? One kind of standardization is having a single file format for a certain “kind/type” of data. For eg. Ogg for audio, or ODF for office-applications.
The kind of standardization that you are suggesting is.. have one single file format for all possible kinds/types of data! But the question is.. Why? What problem do you solve by doing that? What extra feature or facility becomes available with that ?

Now you agree that having one single application that will handle (read/write/edit) ALL kinds of data has is not the right way to do things. I mean, if you want such a thing, then you can just raise your level of abstraction to look at an OS+apps (or distributions in linux), which you view as a big modular application that can handle simply any kind of data/file that you throw at it.

So, if we have different applications, rather more specialized apps, like an multimedia editor for multimedia data, and spreadsheet or office apps for office like data, then whats the point of having a single file format, cos the multimedia app is not going to be able to allow you to do your spreadsheet stuff, or the office app allow you to do video editing, right?

You could say that you can easily “embed” video content in your spreadsheet, but in that case your spreadsheet app uses some components which themselves understand how to deal with that video content. The spreadsheet app simply talks to component through some defined interface, but doesnt have to worry about details of that format itself.

What has been pointed out again by Radical is the concept of shared metadata for all documents which is implemented using the Baegle..

Actually, i didnt say *shared* metadata, i said single defined query-able _format_ of metadata. In other words, any app should be able to query/get details of metadata of any file that it is interested in a defined standard manner. Say, you could have that in xml. I mentioned beagle as an example of what can be achived today, w/o having to change file formats. Basically, beagle understands tons of file formats, and generates a mini-db of all the metadata of the files that it indexes. Now you could hack/alter it to allow you(any app) query the metadata in a defined manner, say a webservice.

One more thing, you mentioned that your motivation for this is that you want to avoid a company having to pay licensing fees to so many different companies for different s/w (like pdf, xls-etc), but for that i’d suggest look at free(dom)-software, rather than this route.

………………
Now the other stuff about ‘ls’ etc, i think thats a completely different use case, its infact a _use-case_, from a users POV! Its a policy in my opinion, i mean, if you are building your system then you could very well choose to have whatever policy that you want, and that would be the ‘behavior’ of your system. Perfectly legal i think

So, lets not mix this with the single file format stuff.

By Ankit on 06.22.06 3:42 pm

[...] Some more controversy over a universal document format comes Microsoft’s way (and thankfully not mine ). Considering the amount of discussions and confusion over my earlier post, I can totally understand the kind of problems Microsoft is going through trying to unify their document formats. However, this time, the battle is between Microsoft and Adobe. Microsoft would want to provide the PDF format as an output option for its Office documents, but Adobe seems unimpressed. [...]

By The Collective » What Microsoft Lacks is a Universal Document Format? on 06.28.06 11:02 am