Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

Does anyone know more about them?
Thomas Weber


--

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com


_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Bob Zawalich-3

If you have already created a hash or array object, these appear to work. Here is an example using an array. “Hash” or “Array” should be replaced by your created Hash or Array object.

 

 

-----Original Message-----
From: Plugin-dev <[hidden email]> On Behalf Of Thomas Weber
Sent: Tuesday, October 2, 2018 6:05 AM
To: A mailing list for Sibelius plug-in developers <[hidden email]>
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

 

Does anyone know more about them?

Thomas Weber

 

 

--

 

Notengrafik Berlin GmbH

HRB 15007

 

UstID: DE 289234097

Geschäftsführer:

Thomas Weber und Werner J. Wolff

 

fon: +49 30 220661685

 

Leuschnerdamm 13

10999 Berlin

 

notengrafik.com

 

 

_______________________________________________

Plugin-dev mailing list

[hidden email]

http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com


_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
Ah – I thought that Array and Hash in the documentation were namespaces, not variable names.  Obvious, when I think about it now.

Many thanks!
Thomas



Am 03.10.2018 um 22:55 schrieb Bob Zawalich:

If you have already created a hash or array object, these appear to work. Here is an example using an array. “Hash” or “Array” should be replaced by your created Hash or Array object.

 

 

-----Original Message-----
From: Plugin-dev [hidden email] On Behalf Of Thomas Weber
Sent: Tuesday, October 2, 2018 6:05 AM
To: A mailing list for Sibelius plug-in developers [hidden email]
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

 

Does anyone know more about them?

Thomas Weber

 

 

--

 

Notengrafik Berlin GmbH

HRB 15007

 

UstID: DE 289234097

Geschäftsführer:

Thomas Weber und Werner J. Wolff

 

fon: +49 30 220661685

 

Leuschnerdamm 13

10999 Berlin

 

notengrafik.com

 

 

_______________________________________________

Plugin-dev mailing list

[hidden email]

http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com



_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Bob Zawalich-3
No, it was really not obvious. I figured that you would need to get your variable in there somewhere so I tried what ended up working. 

I have never used these converters in a plugin but I always use dictionaries and sparse arrays when I can. Dictionaries in particular were a game changer and they are great for sorting array elements and for removing duplicates. 

Bob

On Oct 6, 2018, at 7:42 AM, Thomas Weber <[hidden email]> wrote:

Ah – I thought that Array and Hash in the documentation were namespaces, not variable names.  Obvious, when I think about it now.

Many thanks!
Thomas



Am 03.10.2018 um 22:55 schrieb Bob Zawalich:

If you have already created a hash or array object, these appear to work. Here is an example using an array. “Hash” or “Array” should be replaced by your created Hash or Array object.

 

<image001.png>

 

-----Original Message-----
From: Plugin-dev [hidden email] On Behalf Of Thomas Weber
Sent: Tuesday, October 2, 2018 6:05 AM
To: A mailing list for Sibelius plug-in developers [hidden email]
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

 

Does anyone know more about them?

Thomas Weber

 

 

--

 

Notengrafik Berlin GmbH

HRB 15007

 

UstID: DE 289234097

Geschäftsführer:

Thomas Weber und Werner J. Wolff

 

fon: +49 30 220661685

 

Leuschnerdamm 13

10999 Berlin

 

notengrafik.com

 

 

_______________________________________________

Plugin-dev mailing list

[hidden email]

http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com



_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
As the old style data structures are still used by the API (e.g. in ReadTextFile()), it's good to know how to convert them if needed.

So far I've not been tempted to explicitly create the old structures myself either, but they have at least one interesting feature that might be useful sometimes: They "auto-expand" when you index them with a non-existant key, provdiding a new object that you can use immediately. So code like this works:

    h = CreateHash();
    h.a.b.c = 1;

But they also have dangerous features, e.g.

    Trace(CreateHash() = null);

prints "true".  I developed the habit of testing for null in the reverse order in ManuScript, e.g.

    null = CreateHash()



Am 06.10.2018 um 19:08 schrieb Bob Zawalich:
No, it was really not obvious. I figured that you would need to get your variable in there somewhere so I tried what ended up working. 

I have never used these converters in a plugin but I always use dictionaries and sparse arrays when I can. Dictionaries in particular were a game changer and they are great for sorting array elements and for removing duplicates. 

Bob

On Oct 6, 2018, at 7:42 AM, Thomas Weber <[hidden email]> wrote:

Ah – I thought that Array and Hash in the documentation were namespaces, not variable names.  Obvious, when I think about it now.

Many thanks!
Thomas



Am 03.10.2018 um 22:55 schrieb Bob Zawalich:

If you have already created a hash or array object, these appear to work. Here is an example using an array. “Hash” or “Array” should be replaced by your created Hash or Array object.

 

<image001.png>

 

-----Original Message-----
From: Plugin-dev [hidden email] On Behalf Of Thomas Weber
Sent: Tuesday, October 2, 2018 6:05 AM
To: A mailing list for Sibelius plug-in developers [hidden email]
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

 

Does anyone know more about them?

Thomas Weber

 

 

--

 

Notengrafik Berlin GmbH

HRB 15007

 

UstID: DE 289234097

Geschäftsführer:

Thomas Weber und Werner J. Wolff

 

fon: +49 30 220661685

 

Leuschnerdamm 13

10999 Berlin

 

notengrafik.com

 

 

_______________________________________________

Plugin-dev mailing list

[hidden email]

http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com



_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Bob Zawalich-3

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer), and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing. It is also possible to create global Sparse Arrays and dictionaries, which can endure for a Sibelius session, even if the plugin stops, so they have significant advantages.

 

For sorting arrays for dialogs these days, I almost always load objects into a dictionary, using an appropriate key, and  then fill an array from the dictionary, as in

 

for each Name name in dict

{

            str = dict[name];

            arr[arr.NumChildren] = str;

}

 

For key, I will do something like this:

 

            KeyForBarObjInDict "(obj, fIncludeVoice) {

// staff bar pos voice

 

key = utils.MakeNDigitNumber(obj.ParentBar.ParentStaff.StaffNum, 4) & ';' &

            utils.MakeNDigitNumber(obj.ParentBar.BarNumber, 5) & ';' &

            utils.MakeNDigitNumber(obj.Position, 5);

 

if (fIncludeVoice)

{

            key = key & ';' & obj.VoiceNumber;

}

 

return key;}"

 

where I force numbers like staff and bar numbers to be a fixed number of digits so that the aggregate key can be sorted as a string.

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

 

From: Thomas Weber <[hidden email]>
Sent: Tuesday, October 9, 2018 7:37 AM
To: Bob Zawalich <[hidden email]>
Cc: A mailing list for Sibelius plug-in developers <[hidden email]>
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

As the old style data structures are still used by the API (e.g. in ReadTextFile()), it's good to know how to convert them if needed.

So far I've not been tempted to explicitly create the old structures myself either, but they have at least one interesting feature that might be useful sometimes: They "auto-expand" when you index them with a non-existant key, provdiding a new object that you can use immediately. So code like this works:

    h = CreateHash();
    h.a.b.c = 1;

But they also have dangerous features, e.g.

    Trace(CreateHash() = null);

prints "true".  I developed the habit of testing for null in the reverse order in ManuScript, e.g.

    null = CreateHash()



Am 06.10.2018 um 19:08 schrieb Bob Zawalich:

No, it was really not obvious. I figured that you would need to get your variable in there somewhere so I tried what ended up working. 

 

I have never used these converters in a plugin but I always use dictionaries and sparse arrays when I can. Dictionaries in particular were a game changer and they are great for sorting array elements and for removing duplicates. 

Bob


On Oct 6, 2018, at 7:42 AM, Thomas Weber <[hidden email]> wrote:

Ah – I thought that Array and Hash in the documentation were namespaces, not variable names.  Obvious, when I think about it now.

Many thanks!
Thomas



Am 03.10.2018 um 22:55 schrieb Bob Zawalich:

If you have already created a hash or array object, these appear to work. Here is an example using an array. “Hash” or “Array” should be replaced by your created Hash or Array object.

 

<image001.png>

 

-----Original Message-----
From: Plugin-dev [hidden email] On Behalf Of Thomas Weber
Sent: Tuesday, October 2, 2018 6:05 AM
To: A mailing list for Sibelius plug-in developers [hidden email]
Subject: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

The documentation states, that "old style" Hash and Array objects can be converted to Dictionaries and SparseArrays, respectively, with the Hash.ConvertToDictionary() and Array.ConvertToSparseArray() functions.  However, those don't seem to exist – Sibelius tells me "Field Array not found" or "Field Hash not found. They aren't in the global namespace or the Sibelius namespace, either.

 

Does anyone know more about them?

Thomas Weber

 

 

--

 

Notengrafik Berlin GmbH

HRB 15007

 

UstID: DE 289234097

Geschäftsführer:

Thomas Weber und Werner J. Wolff

 

fon: +49 30 220661685

 

Leuschnerdamm 13

10999 Berlin

 

notengrafik.com

 

 

_______________________________________________

Plugin-dev mailing list

[hidden email]

http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com




_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com

 

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

 

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),


Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes.

Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):
{
    ListAll "(array) {
        i = 0;
        for each entry in array {
            Trace(i & ' ' & entry);
            i = i + 1;
        }
        Trace('');
    }"

    Run "() {
        a=CreateArray();

        a[0] = 'a';
        a[1] = 'b';
        ListAll(a);

        a.foo = 'c';
        a.bar = 'd';
        ListAll(a);

        a[2] = 'cc';
        Trace(a.foo);
        a.foo = 'ccc';
        Trace(a[2]);
        Trace(a.foo);
    }"
}
Which prints:

0 a

1 b


0 a

1 b

2 c

3 d


cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.


Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.


and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.


Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?


 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.


I'd imagine!  The documentation on technical language details is pretty "sparse".


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Bob Zawalich-3

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber <[hidden email]>
Sent: Wednesday, October 10, 2018 2:09 AM
To: Bob Zawalich <[hidden email]>
Cc: 'A mailing list for Sibelius plug-in developers' <[hidden email]>
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),


Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes.

Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{
    ListAll "(array) {
        i = 0;
        for each entry in array {
            Trace(i & ' ' & entry);
            i = i + 1;
        }
        Trace('');
    }"

    Run "() {
        a=CreateArray();

        a[0] = 'a';
        a[1] = 'b';
        ListAll(a);

        a.foo = 'c';
        a.bar = 'd';
        ListAll(a);

        a[2] = 'cc';
        Trace(a.foo);
        a.foo = 'ccc';
        Trace(a[2]);
        Trace(a.foo);
    }"
}

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.


Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?



 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.


I'd imagine!  The documentation on technical language details is pretty "sparse".



-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email]
Sent: Wednesday, October 10, 2018 2:09 AM
To: Bob Zawalich [hidden email]
Cc: 'A mailing list for Sibelius plug-in developers' [hidden email]
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),


Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes.

Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{
    ListAll "(array) {
        i = 0;
        for each entry in array {
            Trace(i & ' ' & entry);
            i = i + 1;
        }
        Trace('');
    }"

    Run "() {
        a=CreateArray();

        a[0] = 'a';
        a[1] = 'b';
        ListAll(a);

        a.foo = 'c';
        a.bar = 'd';
        ListAll(a);

        a[2] = 'cc';
        Trace(a.foo);
        a.foo = 'ccc';
        Trace(a[2]);
        Trace(a.foo);
    }"
}

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.


Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?



 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.


I'd imagine!  The documentation on technical language details is pretty "sparse".



-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:
a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());
Which will Trace this:
a
{
	b "bar"
	{
		 "foo"
	}
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
	Text {
		Title "text 1"
	}
	Text {
		Title "text 2"
	}
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas




Am 11.10.18 um 17:31 schrieb Thomas Weber:
Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Alexander Plötz

Hi Thomas,

can you give a bit more detail about what you are trying to do? Before I (for better or worse) stopped ManuScript work, I dabbled in a few things that might or might not overlap with what you are asking about. But your question is rather abstract, and I always did my plug-in work on a very pragmatic level, so knowing more could trigger a few memories on my side.

Alex

 

From: Plugin-dev [mailto:[hidden email]] On Behalf Of Thomas Weber
Sent: Sonntag, 28. Oktober 2018 19:02
To: Bob Zawalich
Cc: 'A mailing list for Sibelius plug-in developers'
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:

a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());

Which will Trace this:

a
{
  b "bar"
  {
          "foo"
  }
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
  Text {
         Title "text 1"
  }
  Text {
         Title "text 2"
  }
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas

 

 

 

Am 11.10.18 um 17:31 schrieb Thomas Weber:

Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Alexander Plötz
In reply to this post by Thomas Weber

PS: while I don’t remember if I ever created new dialogs during runtime, I do know that I certainly was able to very flexibly control a large number of their properties once I had set up a suitable method library for that purpose. With creative manipulation, this was more or less the equivalent of creating dialogs ad hoc, simply by re-arranging existing dialogs into the form needed at the moment.

 

From: Plugin-dev [mailto:[hidden email]] On Behalf Of Thomas Weber
Sent: Sonntag, 28. Oktober 2018 19:02
To: Bob Zawalich
Cc: 'A mailing list for Sibelius plug-in developers'
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:

a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());

Which will Trace this:

a
{
  b "bar"
  {
          "foo"
  }
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
  Text {
         Title "text 1"
  }
  Text {
         Title "text 2"
  }
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas

 

 

 

Am 11.10.18 um 17:31 schrieb Thomas Weber:

Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
In reply to this post by Alexander Plötz
Hi Alex,

thanks for the reply.  So far, my interest is indeed rather abstract, though I see some applications for creating PLG structures at runtime.

Dialogs can indeed be manipulated/layouted pretty flexibly at runtime.  I'm not sure whether controls can actually be created at runtime, but they can definitely be layouted very flexibly.

I thought about parsing PLG files at runtime, which could be useful mainly for testing purposes.  I'm programming outside of Sibelius, which is simplified by Tido's plgToMSS[1], and I'm writing unit tests using their sib-test framework[2].  So after every modification, unloads/reloads have to be performed for both the plugin and the test code.  To speed up the tedious unload/reload process, a plugin might parse the modified code and replace the old structures with the updated ones.  That has its limits, because functions created at runtime will not be recognized and run by Sibelius.  As large parts of development work are modification of existing functions, that would still help, though.

BTW, for development outside of Sibelius, I am working on syntax highlighting and syntax checking based on tree-sitter, a new syntax highlighting engine that is now part of the Atom editor.  I've not created an Atom package yet that allows straightforward installation because it's still work in progress.  But it's out there on Github for anyone who's interested to try and get it to work on their system, though I can't promise support.  Pull requests are of course always welcome.

Grüße
Thomas




Am 28.10.18 um 19:27 schrieb Alexander Plötz:

Hi Thomas,

can you give a bit more detail about what you are trying to do? Before I (for better or worse) stopped ManuScript work, I dabbled in a few things that might or might not overlap with what you are asking about. But your question is rather abstract, and I always did my plug-in work on a very pragmatic level, so knowing more could trigger a few memories on my side.

Alex

 

From: Plugin-dev [[hidden email]] On Behalf Of Thomas Weber
Sent: Sonntag, 28. Oktober 2018 19:02
To: Bob Zawalich
Cc: 'A mailing list for Sibelius plug-in developers'
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:

a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());

Which will Trace this:

a
{
  b "bar"
  {
          "foo"
  }
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
  Text {
         Title "text 1"
  }
  Text {
         Title "text 2"
  }
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas

 

 

 

Am 11.10.18 um 17:31 schrieb Thomas Weber:

Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685

Leuschnerdamm 13
10999 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Alexander Plötz

Hi Thomas,

sorry for coming back to this so late; I don't know if any of the following is even helpful for you at this point, but here goes:

I did something like dynamic programming exactly once in ManuScript, and probably for some awkward reason, which, in any case, I don't remember anymore. What I did was to put in a placeholder method (I called it Eph), which, as it was part of the initially read source code, could be successfully re-written and called at run time. I remember that this worked reliably, although the general usefulness for me turned out to be negligible.

However, this might mean that you could effectively have "new" methods from scratch during run time by just providing an ample amount of Eph methods and then use those as your ad hoc functions, by way of indirection*. Maybe a similar approach could also work with plg data structures.

 

On the more general problem of the annoying awkwardness of unloading and reloading a changed plugin: at some point I was so fed up with that that I built a whole pseudo-environment that would allow me to deliberately drop out in any method by use of StopPlugin(). The thing about this is (I hope I remember it right): changes that are made in the editor session that comes up as a result of the StopPlugin() method** will be reflected in the currently loaded version of the plug-in immediately. Additionally, the usual wait time on closing the editor is decreased significantly, probably because there is no actual save-to-file happening at that particular moment anymore.

Coming up with such a meta-construction was a bit of work, but once it was in place I basically never had to unload/reload again; "in place" meaning in this case that the necessary methods were a part of my template for any new plug-in, a template that I built from Bob's "Minimum Plugin".

 

Best,

Alex

 

* alternative idea: have one Eph(), and then really go nuts with the overloading

** as opposed to the "proper" way of opening the editor manually while no plugin is running

 

From: Thomas Weber [mailto:[hidden email]]
Sent: Sonntag, 28. Oktober 2018 22:46
To: A mailing list for Sibelius plug-in developers; Alexander Plötz
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Hi Alex,

 

thanks for the reply.  So far, my interest is indeed rather abstract, though I see some applications for creating PLG structures at runtime.

 

Dialogs can indeed be manipulated/layouted pretty flexibly at runtime.  I'm not sure whether controls can actually be created at runtime, but they can definitely be layouted very flexibly.

 

I thought about parsing PLG files at runtime, which could be useful mainly for testing purposes.  I'm programming outside of Sibelius, which is simplified by Tido's plgToMSS[1], and I'm writing unit tests using their sib-test framework[2].  So after every modification, unloads/reloads have to be performed for both the plugin and the test code.  To speed up the tedious unload/reload process, a plugin might parse the modified code and replace the old structures with the updated ones.  That has its limits, because functions created at runtime will not be recognized and run by Sibelius.  As large parts of development work are modification of existing functions, that would still help, though.

 

BTW, for development outside of Sibelius, I am working on syntax highlighting and syntax checking based on tree-sitter, a new syntax highlighting engine that is now part of the Atom editor.  I've not created an Atom package yet that allows straightforward installation because it's still work in progress.  But it's out there on Github for anyone who's interested to try and get it to work on their system, though I can't promise support.  Pull requests are of course always welcome.

 

Grüße
Thomas

 

 

 

 

Am 28.10.18 um 19:27 schrieb Alexander Plötz:

Hi Thomas,

can you give a bit more detail about what you are trying to do? Before I (for better or worse) stopped ManuScript work, I dabbled in a few things that might or might not overlap with what you are asking about. But your question is rather abstract, and I always did my plug-in work on a very pragmatic level, so knowing more could trigger a few memories on my side.

Alex

 

From: Plugin-dev [[hidden email]] On Behalf Of Thomas Weber
Sent: Sonntag, 28. Oktober 2018 19:02
To: Bob Zawalich
Cc: 'A mailing list for Sibelius plug-in developers'
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:

a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());

Which will Trace this:

a
{
  b "bar"
  {
          "foo"
  }
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
  Text {
         Title "text 1"
  }
  Text {
         Title "text 2"
  }
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas

 

 

 

Am 11.10.18 um 17:31 schrieb Thomas Weber:

Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com



_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com

 

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
Hi Alex,

not sure whether I understand everything correctly.  In my experience, all changes to PLG data made at runtime (e.g. functions, dialogs or anything that is listed in the Data section of the plugin editing window) are "permanent" until Sibelius is restarted or the plugin is reloaded.

I don't think dynamic programming is such a good idea.  I feel it would get really hard to track errors.  But reloading modified functions and data might actually be useful.

I didn't quite understand the StopPlugin() approach, though.  Did you write a plugin that would add StopPlugin() calls to other plugins for debugging purposes?

Thomas



Am 01.12.18 um 17:36 schrieb Alexander Plötz:

Hi Thomas,

sorry for coming back to this so late; I don't know if any of the following is even helpful for you at this point, but here goes:

I did something like dynamic programming exactly once in ManuScript, and probably for some awkward reason, which, in any case, I don't remember anymore. What I did was to put in a placeholder method (I called it Eph), which, as it was part of the initially read source code, could be successfully re-written and called at run time. I remember that this worked reliably, although the general usefulness for me turned out to be negligible.

However, this might mean that you could effectively have "new" methods from scratch during run time by just providing an ample amount of Eph methods and then use those as your ad hoc functions, by way of indirection*. Maybe a similar approach could also work with plg data structures.

 

On the more general problem of the annoying awkwardness of unloading and reloading a changed plugin: at some point I was so fed up with that that I built a whole pseudo-environment that would allow me to deliberately drop out in any method by use of StopPlugin(). The thing about this is (I hope I remember it right): changes that are made in the editor session that comes up as a result of the StopPlugin() method** will be reflected in the currently loaded version of the plug-in immediately. Additionally, the usual wait time on closing the editor is decreased significantly, probably because there is no actual save-to-file happening at that particular moment anymore.

Coming up with such a meta-construction was a bit of work, but once it was in place I basically never had to unload/reload again; "in place" meaning in this case that the necessary methods were a part of my template for any new plug-in, a template that I built from Bob's "Minimum Plugin".

 

Best,

Alex

 

* alternative idea: have one Eph(), and then really go nuts with the overloading

** as opposed to the "proper" way of opening the editor manually while no plugin is running

 

From: Thomas Weber [[hidden email]]
Sent: Sonntag, 28. Oktober 2018 22:46
To: A mailing list for Sibelius plug-in developers; Alexander Plötz
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Hi Alex,

 

thanks for the reply.  So far, my interest is indeed rather abstract, though I see some applications for creating PLG structures at runtime.

 

Dialogs can indeed be manipulated/layouted pretty flexibly at runtime.  I'm not sure whether controls can actually be created at runtime, but they can definitely be layouted very flexibly.

 

I thought about parsing PLG files at runtime, which could be useful mainly for testing purposes.  I'm programming outside of Sibelius, which is simplified by Tido's plgToMSS[1], and I'm writing unit tests using their sib-test framework[2].  So after every modification, unloads/reloads have to be performed for both the plugin and the test code.  To speed up the tedious unload/reload process, a plugin might parse the modified code and replace the old structures with the updated ones.  That has its limits, because functions created at runtime will not be recognized and run by Sibelius.  As large parts of development work are modification of existing functions, that would still help, though.

 

BTW, for development outside of Sibelius, I am working on syntax highlighting and syntax checking based on tree-sitter, a new syntax highlighting engine that is now part of the Atom editor.  I've not created an Atom package yet that allows straightforward installation because it's still work in progress.  But it's out there on Github for anyone who's interested to try and get it to work on their system, though I can't promise support.  Pull requests are of course always welcome.

 

Grüße
Thomas

 

 

 

 

Am 28.10.18 um 19:27 schrieb Alexander Plötz:

Hi Thomas,

can you give a bit more detail about what you are trying to do? Before I (for better or worse) stopped ManuScript work, I dabbled in a few things that might or might not overlap with what you are asking about. But your question is rather abstract, and I always did my plug-in work on a very pragmatic level, so knowing more could trigger a few memories on my side.

Alex

 

From: Plugin-dev [[hidden email]] On Behalf Of Thomas Weber
Sent: Sonntag, 28. Oktober 2018 19:02
To: Bob Zawalich
Cc: 'A mailing list for Sibelius plug-in developers'
Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

On my quest to really understand the data structures of ManuScript:  Apparently plg files are parsed into TreeNodes.  I can create those structures in ManuScript at runtime with the help of CreateArray(), e.g.:

a = CreateArray();
b = CreateArray();
b[0] = "foo";
a.b = b;
a.b = "bar";
Trace(a.WriteToString());

Which will Trace this:

a
{
  b "bar"
  {
          "foo"
  }
}

However, plg data can have strucutres like this, e.g. in Dialogs:

Controls {
  Text {
         Title "text 1"
  }
  Text {
         Title "text 2"
  }
}

The same label (here Text) can be used multiple times. Is it possible to create such structures at runtime? So far, I didn't find a way to do that. If that would be possible, it might open some metaprogramming possibilities. I found that it is not possible to create new functions or dialogs at run-time, but plugin analysis with structures that are equivalent to the structures Sibelius uses itself would be quite interesting.

Thomas

 

 

 

Am 11.10.18 um 17:31 schrieb Thomas Weber:

Many thanks for pointing to the TreeNode objects. I've so far not paid attention to them.


Am 11.10.2018 um 00:20 schrieb Bob Zawalich:

At least arrays and hashes and global variables are variants of Treenode objects (see the reference). My understanding of SparseArrays and Dictionaries is that they are totally different implementations, and that dictionaries do non linear lookup.

The biggest hassle with these if is if you want a value from an array, for example, you need to do something like

 

str = “” & arr[1];

num = 0 + arr[2];

 

because otherwise the arr returns an address and often all hell breaks looks.

 

My 4 least favorite ManuScript characteristics are

 

  1. The above
  2. The fact that expressions are evaluated strictly left to right with no operator priority so you have to parenthesize anything remotely complex
  3. The fact that having a // comment in the final line of a routine causes a syntax error
  4. The fact that comparisons using single character literals give the wrong results.

 

There are more, but I will stop here. I have spent many hours fighting with these even though I know they are out there. Sigh.

 

 

 

 

From: Thomas Weber [hidden email] Sent: Wednesday, October 10, 2018 2:09 AM To: Bob Zawalich [hidden email] Cc: 'A mailing list for Sibelius plug-in developers' [hidden email] Subject: Re: [Plugin-dev] Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

 

Am 09.10.2018 um 22:20 schrieb Bob Zawalich:

Dialogs all require “normal” arrays, so they are still useful or at least necessary.

Arrays are quirky objects, though (you need to escape the return values or you pass a pointer),

Some of my observations that you might already be aware of:  It even seems that this "pointer" actually points to two things: Another Array and a value that is used when coercing the entry to a number, string etc.

a = CreateArray();

a[0] = "a";

a[0].foo = "b";

Trace(a[0]);  // prints a

Trace(a[0].foo); // prints b

a[1] = a[0];

a[1] = "c";

Trace(a[1]);  // prints c

Trace(a[1].foo);  // prints b

The same thing can be done with Hashes. Some interesting stuff is also this (not proper array usage you might say, but nevertheless curious):

{     ListAll "(array) {         i = 0;         for each entry in array {             Trace(i & ' ' & entry);             i = i + 1;         }         Trace('');     }"     Run "() {         a=CreateArray();         a[0] = 'a';         a[1] = 'b';         ListAll(a);         a.foo = 'c';         a.bar = 'd';         ListAll(a);         a[2] = 'cc';         Trace(a.foo);         a.foo = 'ccc';         Trace(a[2]);         Trace(a.foo);     }" }

Which prints:

0 a

1 b

 

0 a

1 b

2 c

3 d

 

cc

ccc

This indicates that arrays can be used in "object fashion" (i.e. `a.foo`). Numerical array entries are created for each object property. Changing the value using the numerical key also changes the value under the object style key and vice versa.

 

Like for SparseArrays, if a non-numeric key is used in array style indexing (i.e. `a[i]`), that key will be coerced to a number and the fractional part will be truncated. Hashes (as well as Dictionaries on the other hand) coerce keys to strings and "object style" access (`h.foo`) is equivalent to "hash style" access (`h['foo']`). However, for Dictionaries "object style" assignment can only be used if an entry for the key is already present, while that doesn't matter for Hashes.

 

and very inefficient. Even indexing in an array actually does a linear search, and so does a hash search – it actually does no hashing.

Do you imply that's changed for SparseArrays and Dictionaries?  Are they properly hashing and have sub-linear lookup time?

 

A lot of those other things I think of less as features than as near-fatal flaws. I have spent a lot of time debugging those quirks.

I'd imagine!  The documentation on technical language details is pretty "sparse".

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com



_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com

 

-- 
 
Notengrafik Berlin GmbH
HRB 15007
 
UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff
 
fon: +49 30 220661685
 
Leuschnerdamm 13
10999 Berlin
 
notengrafik.com


-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685
Friedrichstraße 23a
10969 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com
Reply | Threaded
Open this post in threaded view
|

Re: Hash.ConvertToDictionary() and Array.ConvertToSparseArray()

Thomas Weber
In reply to this post by Thomas Weber
Am 28.10.18 um 22:45 schrieb Thomas Weber:
I thought about parsing PLG files at runtime, which could be useful mainly for testing purposes.  I'm programming outside of Sibelius, which is simplified by Tido's plgToMSS[1], and I'm writing unit tests using their sib-test framework[2].  So after every modification, unloads/reloads have to be performed for both the plugin and the test code.  To speed up the tedious unload/reload process, a plugin might parse the modified code and replace the old structures with the updated ones.

It turns out that a method can only be manipulated before the plugin runs it for the first time. Once a method ran, Sibelius will not re-load it until the user runs the plugin again. Like any other global variable, method manipulations will persist between plugin runs.

-- 

Notengrafik Berlin GmbH
HRB 15007

UstID: DE 289234097
Geschäftsführer:
Thomas Weber und Werner J. Wolff

fon: +49 30 220661685
Friedrichstraße 23a
10969 Berlin

notengrafik.com

_______________________________________________
Plugin-dev mailing list
[hidden email]
http://lists.avid.com/mailman/listinfo/plugin-dev_lists.avid.com