JavaScript Interpreter Notes
============================

(Update: John Albano 10/21/02)

There are some circumstances under which a particular regular expression applied to a particular body of text can result in either an extremely long or seemingly infinite processing loop.

In order to prevent this, we have added a limit to the number of recursive calls to matchRENodes() that can be made for a single call to MatchRegExp().

This new logic is controlled by the DREAMWEAVER_LIMIT_REG_EXP_RECURSION define in regexp.c.

(Update: Steve Newman 3/19/01)

I've just upgraded from RC2 to RC3 of the JavaScript 1.5 interpreter.
This should fix the bug with anonymous nested functions (mentioned
below). At some point, we should upgrade to the final version of
JS1.5 (once Netscape releases it).

Here is the procedure I followed to install RC3, it should be applicable
to any future releases unless/until Netscape makes a major reorganization
to the code.

1. Get a source code bundle from Mozilla's FTP server. (You can also get
the code from CVS, but presumably it's better to get a "stable" release.)

2. The root folder should contain a README file, a CVS folder, and a src
folder. Check the README for any news, and then discard it and the CVS
folder.

3. Within the src folder, discard any CVS folders or .cvsignore files.
(These appear at multiple levels in the folder hierarchy, you need to
walk down each subfolder to locate them.) Also discard these entries
from the top level of the src folder:

:config:
:editline:
:fdlibm:
:macbuild:
:perlconnect:
:sh:

js.mdp
js3240.rc
lock_SunOS.s
MANIFEST
plify_jsdhash.sed
resource.h
SpiderMonkey.rsp
win32.order
*.mk
Makefile.*
makefile.*

4. Copy the _DW folder and the DW_decls.h file from the
ThirdParty/JavaScript_1_5 folder in the Dreamweaver source.

5. Make a copy of src/jsinterp.h, and name it jsinterp2.h (leaving
it in the same folder as jsinterp.h).

6. Search for the word "DREAMWEAVER" in the ThirdParty/JavaScript_1_5
folder in the Dreamweaver source. Look at each match, see what change
it describes, and make an equivalent change to the new source.

7. Insert this line:
#include "DW_decls.h"

near the top of every .c file in the new Netscape code.

At this point, you can replace the contents of the
ThirdParty/JavaScript_1_5 folder in the Dreamweaver source, with
the contents of the "src" folder.

(Original notes: Steve Newman 2/22/01)

This file contains some notes on the JavaScript interpreter that we
use in Dreamweaver. I originally created it while cleaning up some
bugs that were introduced with the move to JavaScript 1.5 at the
outset of Vertigo (Dreamweaver 5).

Dreamweaver and Ultradev use Netscape's C-based JavaScript
interpreter, code-named SpiderMonkey. Information is available
on this page:

http://mozilla.org/js/

Dave George and NJ did the original work to install the interpreter back
in '98. Prior to Vertigo, we used an old version of the code (licensed
directly from Netscape) which implemented JavaScript version 1.2. In
December 2000, Charles McBrian downloaded the 1.5-rc2 version of the
code from Netscape's FTP server and incorporated it into the Vertigo
codebase. This code is in ThirdParty/JavaScript_1_5; the old code was
in ThirdParty/NscpJavaScript.

We've made a handful of tweaks to Netscape's source. All of these
changes are flagged with the keyword "DREAMWEAVER". The tweaks are
mostly to fix build problems or to expose private functions which we
need; there is one wierd patch to the WITH lookup code which is
discussed below (and in a comment on the patch itself).

Note that in the 1.2 source, we made one tweak each to
Include/prtypes.h and to Source/os/win32.h. We aren't using these
files in the 1.5 interpreter, so I didn't have any way to carry
those tweaks across. (Search for "DaveG" in the NscpJavaScript
folder to see them.) Doesn't seem to be a problem. I also didn't
carry across a change to a _control87 call in jsnum.c which didn't
seem applicable in the 1.5 codebase.

Dreamweaver defines many custom classes of JavaScript objects.
In some (all?) of these, we define additional properties, but
don't support iteration over those properties. For example, you
can't iterate over the properties of a JSObjDocument that come
from named nodes in the DOM. To fix this, I think we'd need to
find all of the places where we define a new class, and ensure
that we define an iteration function for the class which matches
the getter function.

If you're going to do any work on the JavaScript interpreter, and
in particular if you're trying to upgrade us to a newer version,
a good first step is to search for "DREAMWEAVER" in the code we're
currently using (ThirdParty/JavaScript_1_5).

NOTE: as of this writing (2/22/01), there is a bug with anonymous
nested functions that breaks the test harness. It appears that
this bug was fixed by Netscape subsequent to the 1.5-rc2 release
that we're using. I am corresponding with a Netscape engineer
about the problem and will get a fix into our copy of the code
shortly. We may want to try to get a newer copy of the code from
Netscape before shipping Vertigo. (UPDATE 3/23/01: this should
have been fixed by the move to 1.5-rc3; I haven't checked.)

Additional notes from DaveG
===========================

- You're right that getting and setting a property is a messy piece of code.
If you look at the types that are built into the interpreter, such as
jsarray.c, you'll see that overriding the getProperty and setProperty
methods is not only kosher - it's the way that they define their own types.

- I've been working with a version of the JS interpreter that doesn't
support LiveConnect, and the property getting/setting is still pretty darn
complicated. Part of the complexity seems related to their caching scheme,
which was presumably implemented for performance reasons.

- A jsval can hold a JSString pointer, or it might hold an integer, a
boolean, a pointer to a JSDouble, or a pointer to a JSObject. As you
hinted, the two low-order bits are used to remember the type thing stored in
the jsval.

- If you stop the debugger in the middle of a Javascript script, it's often
useful to know the contents of the JS stack. There's a field on the
js_context global (I think it's called "frame" or something like that) that
stores a linked list of stack frames.

- You can get the type of a (JSObject *) in the Visual C++ quickwatch window
by typing JS_GetClass(obj).

Random debugging tips
=====================

General:

In the Extension Debugger code, I've defined a couple of native
functions which are useful for debugging interpreter problems.
(Note that the Extension Debugger is not checked in as of this
writing.)
debug_cDebugger(message); // Drops into the C-level debugger
// with the given message string.
// (Message string is optional.)

debug_dumpObj(obj); // Writes a list of obj's fields
// to the trace output.

These are handy to get into the C debugger at a specific point
during JavaScript execution.

Looking at JSObjects in the debugger:

This is a pain. A few useful things to know:
A JSObject's map field is really a JSScope*, you can coerce it
to this in the debugger to see more information.
To find out what string a JSAtom corresponds to, look at the
key part of the hash entry field. This is typically a jsval,
which in turn is typically a JSString* plus a fixed offset (I
think the offset is 4, but it might be 8).

Understanding how properties are accessed in a JSObject:

This is really complicated. For each JSClass, there is a table
of function pointers which implement operations such as "get
property" and "set property". These are often overridden; we
override these functions in many of the classes that DW defines.
There is another set of function pointers in the object's map->ops
field. These seem to be lower-level operations, and are usually not
overridden. (I have a hunch that they were introduced for use by
LiveConnect.) And there's a third small table of function pointers
in JSScope's "ops" field; this seems to be related to hash table
lookup, and is probably never overridden.
The upshot is that you should pay very close attention when trying
to step through a get-property call. There are many levels of
function call, involving all three of the function pointer tables
mentioned above, and a complicated dance between "lookup" and "get"
operations. (As far as I can tell, "lookup" operations determine
whether an object has a certain property, and get metadata about
the property; "get" operations actually fetch the value.)

Many of the classes defined by Dreamweaver publish dynamically
defined properties. For example, a document object publishes a
property for each named DOM node. This is implemented by
overriding the get-property operation in the JSClass's function
table. This seems to not be "kosher" -- I gather that you're
supposed to actually create a record for each property in the
object's hash table. The way we do things works for operations
that call "get" directly, but not for operations that use "lookup".
I think this means that various uncommon operations, such as
adding or removing properties, won't always work properly on
(say) document objects. It also breaks WITH lookup into our
objects, which we've patched in the JavaScript interpreter.
To implement this "correctly", we'd either need to override
various function pointers in the JSObjectOps table for all of
our classes, or else we'd need to create a property record for
every one of these dynamically-defined properties. Right now
that seems like way too much work for little apparent gain.
Debugging problems with WITH:

WITH statements are implemented by the JSOP_ENTERWITH opcode in
jsinterp.c. This creates an object and pushes it onto a stack
of scope objects. For some reason SpiderMonkey wraps a special
object around the object used in the WITH; all lookup operations
pass through this wrapper object and are forwarded along to the
original object. We had to patch this to get around the
problem with dynamically defined attributes in Dreamweaver objects
not being done correctly.