I was not prepared for the amount of responses I got to my last article. There really are two camps – and they sit around fires making pointy sticks.
The most constructive was from Rob Sanheim with this post, which got me rather excited.
Now before I code anything, I write a set of testcases. Testcases let you play with your API and sort out the semantics and ‘feel’ before you get too far into the project to change it. Testcases – Just Do It. (You know who you are).
So within 10 seconds of reading Rob’s post I popped the latest prototype.js into my test system and had my result.
I used to have 1 extra dangleberry named ‘extend’ in every JavaScript Object. At least with that I had a fighting chance.
With prototype.js version 1.4.0 I have 33.
Here they are in all their tentacled glory.
each
all
any
collect
detect
findAll
grep
include
inject
invoke
max
min
partition
pluck
reject
sortBy
toArray
zip
inspect
map
find
select
member
entries
_reverse
_each
clear
first
last
compact
flatten
without
indexOf
Now, I’m not complaining but, oh hang on – I am.
My previous fix for my rendering pipeline is completely out the window.
Now on top of my previous complaint, I’m wondering what the performance hit is for always having 33 objects to check against on every object slot access? In theory if the slot manager uses a good hashtable this won’t be more significant than having just one object.
I have a library that can measure this, but prototype.js breaks it.
Its 3 o’clock and on the dark side of breakfast – for now I will sleep on this.
James McParlane's Blog 
Here’s a comment on this, originally posted by me on the rails-spinoffs mailing list (which is for Prototype and script.aculo.us), there was a discussion on this recently:
—-8<—-snip—-
The problem here is really not Prototype extending JavaScript built-in objects. JavaScript is a dynamic language and you’re not forbidden from extending the built-ins to become more programmer-friendly.
It really all boils down to one thing:
The for-in loop.
This is a language construct that iterates over all properties of an
object. Because JavaScript is a dynamic language where you can add/change
Object prototypes at all times, this construct doesn’t guarantee the
kind of properties which are returned.
According to the ECMAScript Language Specification:
"Enumerating the properties of an object includes enumerating properties of its prototype".
So the problem is that the for-in loop is just not used with that in mind when it’s called upon for stuff like iterating over the elements of an array.
(Sadly, the JavaScript "DontEnum" attribute can’t be set manually, so it can’t be used to mask/hide individual properties (like the addtional functions on the Object prototype): "DontEnum: The property is not to be enumerated by a for-in enumeration").
IMHO, trying to not to use extensions for built-in objects (where
they really make sense, see Prototype’s Enumerables) to avoid errors because of "misinformed programming" in other libraries is not using the JavaScript language for what it is.
Note that even Brendan Eich, the "father of JavaScript" recommends that if you want extensions on built-in objects, than just do it (as it’s meant for that). (He also acknowledges the for-in loop problem and is in favor of adding something to the next version of JavaScript so you can define DontEnum properties, that won’t show in for-in loops.)
—-8<—-snip—-
-Thomas
All good points – it does boil down to the behavior of the for-in loop.
The only recourse for the ‘its OK to extend built-in objects’ camp is to blame for-in, which makes sense because the only other current solution is to stop extending the Object and Array built-in objects. 🙂
I do believe the problem is with JavaScript, and that’s where the ultimate solution must come from – but that’s not going to resolve the issue any time soon.
In non-prototype based languages is customary to derive your new behavior from the bultins via inheritance. Going into the low level libraries and modifying them to suit your purposes is valid – if you don’t require your program to run on anyone else’s machine. 🙂
I don’t believe its "misinformed programming" to want to be able to combine JavaScript libraries together – and certainly that’s what prototype.js and all its derived libraries is achieving, and very well.
Its when you try to mix prototype.js with libraries that use for-in that it all goes horribly avocado shaped.
Another possibility is Array behavior. Perhaps for-in should only enumerate items that have been specifically added via an A[n] = x or A.push(x) ?
If declaring shenanigans on for-in is taken seriously, and I am taking this seriously, this would require a recoding of any library that uses for-in and an education of JavaScript coders. A simple re-factoring would use two arrays, one for fast hash lookup and a numericly indexed array of references to the other objects for enumeration. This just may not be practical for many – I am looking into this myself for my library just so I can interoperatte with prototype.js
This approach will be the topic of my next post.
Two things are certain
1) The current resurgence in JavaScript is because the kinds of modern re-use strategies normally reserved for higher order languages is being applied.
2) If we want to be able to combine JavaScript libraries together, we need to know which camp they fall into – because they simply won’t work if you mix them.
James,
Thanks for writing about this. You argue your point well. I am just getting into JavaScript (I’ve really been saying that for about 4 years while I work my day job debugging C), and as AJAX is starting to get interesting, I’m looking into what JavaScript libraries to adopt. The hype surrounding Prototype makes it seem like a no-brainer, but this issue is making me think hard about it.
Suppose you find a library that does something you need, and which plays nice with Prototype, and you decide to use it in your app, only to find that the next release of the library breaks when combined with Prototype? You can’t upgrade! If you’re using SlayerOffice-style remote hosting of that library, which gets upgraded in place by the maintainer, you’re just screwed!
A bit late to the party, but I stumbled here so I’d might as well say my piece.
"The only recourse for the ‘its OK to extend built-in objects’ camp is to blame for-in, which makes sense because the only other current solution is to stop extending the Object and Array built-in objects. :)"
There is another camp, the camp that says that adding to Object.prototype is a bit no-no, but extending Array.prototype (or String.prototype, &c.) is just fine. I’m in that camp.
To be frank, back before Prototype 1.4, I wouldn’t have gone near it for that exact reason: messing with Object.prototype isn’t clever.
However, beyond not being able to use for-in on arrays (something I’ve never done myself), I can’t see what your objection is.
Yes, what you describe is the ‘Appeasement’ compromise between the camps which goes something like this.
1) Don’t override ‘Object’.
2) Do what you like to ‘Array’
3) If you want an associative array, use ‘Object’ instead.
This has the effect of allowing one type to be overridden to hell and back while reserving another type to remain pristine.
It seems to work.
Of course Prototype still breaks older code that does not take this new real-politic into account, but its easier to backport these libraries with at least the breathing room of a clean ‘Object’.
It’s not really ‘Appeasement’, as you describe it. Unlike Object, Array is no more fundamental than any other object. That makes it safe to extend. Object, on the other hand, *is* fundamental, which is what makes extending it unsafe. It’s not a compromise.
Note though that Array is *only* meant to be used for numeric indexing. It’s not meant to be used as an associative array. I must admit that I’ve never came across any code in the wild using Array in such a manner, though if I did I’d have some serious words for the programmer as their code would be breaking the principle of least surprise. It’s no better than using an instance of Regex, String, or some other object without right semantics as an associative array.
This scheme doesn’t just work: it’s how Brendan Eich intended it to be used. Realpolitik (defined as ‘Governmental policies based on hard, practical considerations rather than on moral or idealistic concerns. Realpolitik is German for the politics of reality and is often applied to the policies of nations that consider only their own interests in dealing with other countries’) this is not. The Realpolitik approach would be to derive a Hashtable object from Object to act as the basis for associative arrays like I’ve seen some do. Recognising that Object and only Object is special and that derived objects with differing semantics (such as Array, String, Regex, and non-core object) should *not* be treated as associative arrays is about a puritan as you can get.
All good points and thank-you all for your high quality comments.
I have seen lots of code in the wild that does use for(in) to iterate an Array. It does seem to be a natural and forgivable thing for a programmer to do seeing the term is normally taught is "Associative Array", not "Associative Object".
Reference Sites Give Examples
http://www.quirksmode.org/js/associative.html
And yes Brendan Eich may have intended Arrays to be only numeric, but the reality is they do more, and people have used them to the fullest extent possible.
It is arguable that this exploration and pushing of the JavaScript envelope has made this whole Web 2.0 thing possible.
I won’t blame people for being creative.
As for my "Realpolitik" reference, depends on which side you are on, one sides practicality can be another’s idealism. My position is that allowing ‘Array’ to be overridden and leaving Object pure, is a practical solution that seems to work and makes most people happy. At least we can have Associative Objects and back-port code, so yes, we agree that the Realpolitik solution is to use Object to create an Associative array.
If we deferred the decision to the purist idealists then we get this http://www.andrewdupont.net/2006/05/18/javascript-associative-arrays-considered-harmful/ , the non-numeric Array abstinence philosophy.
This is probably a good thing if we can get everyone to stop using Arrays in this way. The reality is that this is probably not going to happen.
So do we hope and wait till this is ‘fixed’ in JS2, but that means that we have 4-5 years now of broken JavaScript.
I’m not even sure it will be fixed.
"Since Edition 4 is aiming for backward compatibility, we are not redefining iteration over Array to differ (to iterate values). But we are in fact thinking of better ways to expose value iteration than for-each. More in a new blog post."
http://weblogs.mozillazine.org/roadmap/archives/2006/02/js_and_python_news.html
So what can we do?
1) Educate – Conversations like this make people aware that there is an issue, and help explain how a "best practice" compromise is reached.
2) Write tools – A plugin for Firefox for developers that warns against non-numeric Arrays? Anyone writing debug plugins should probably add this as a warning. JavaScript/HTML editors should warn if an array is detected being setup as Non-numeric.
3) Compromise – I Think Dean Ewards explains it best.
"Extending native objects is a good way to provide backward compatibility for older browsers (e.g. providing push/pop for IE5.0). If we cant do this then we are stuck with only the basic features of the language.
Extending Object.prototype is another matter as then we are left with no object that we can use as a hash."
‘I have seen lots of code in the wild that does use for(in) to iterate an Array. It does seem to be a natural and forgivable thing for a programmer to do seeing the term is normally taught is "Associative Array", not "Associative Object".’
It’s very much dependent on your background. I started with Commodore BASIC and 6510 assembly, then BBC BASIC and ARM assembly, then C, then Perl, then Java and C++, and so on. My background makes me differentiate between numeric arrays and associative arrays and assume an array is numeric unless told otherwise. But the biggest element of confusion is that people aren’t familar with the idea of Prototypes and will tend to treat them like objects in class-based systems. There lies a world of pain!
‘And yes Brendan Eich may have intended Arrays to be only numeric, but the reality is they do more, and people have used them to the fullest extent possible.’
But they only work that way by accident of the fact Object is the prototype of Array. As I said, if one wants to treat and instance of Array as an associative array, there’s nothing stopping one from using instances of other objects is an inappropriate manner. Using an instance of String as an associative array could hardly be described as ‘[using] them to the fullest extent possible’, I think you’ll agree.
‘This is probably a good thing if we can get everyone to stop using Arrays in this way. The reality is that this is probably not going to happen.’
The alternatives are to boycott the use of code that behaves like this in projects you work on and to educate coworkers who abuse Array and Object. I’ve had to do this in the past with coworkers who, not knowing the consequences, would add methods to Object.prototype. Do it politely and with examples, and people usually come around.
I rarely iterate over arrays these days myself. Nowadays, I check if the JS1.6 iteration methods are present on Array.prototype and if not extend it with the relevant code and use those. It’s much nicer.
What you’re describing as the "Realpolitik" position is, best as I can tell, the opinion of the vast majority of the apparent purists. I consider myself one of them, and agree with what Andrew has to say. The few who believe extending _any_ core object is a bad idea mostly don’t get prototypes very well.
‘I won’t blame people for being creative.’
There’s creative and then there’s shortsightedness. Modifying Object.prototype or treating Array as a hash is akin to writing self-modifying code for a processor with a cache: it will bite you.
Oh, and why does the captcha ask for the codes twice?
My path to enlightenment was Sinclair Z80, Vic20, C64, PDP-11, Amiga, 🙂
On the Education route, I’ve tried to call people to action.
https://blog.metawrap.com/blog/June6thIsJavaScriptArrayAndObjectprototypeAwarenessDay.aspx
Would appreciate your suggestions.
As for the captcha, This blog works in mysterious ways. 🙂