Custom Query (121 matches)

Reported by idadesub@…, 5 years ago.

Hello,

I've started to play around with porting code over to python 3, and since I use yaml, I had to port it over as well. So, here's the patch :) It's pretty heavyweight, though, because of the change of "str" to be unicode. Since there are many incompatible changes, I'd recommend creating a branch for this.

Here's some other notes that you ought to be aware of:

• In order to do from error import * you now have to specify the full path to the module from yaml.error import *, explicitly specify what you want to import with from .error import YAMLError, or from . import error. I went with from yaml.error import * as that let me touch less code.

• Removed:

try:
    set
except NameError:
    from sets import Set as set

• Removed utf_*_decode functions from yaml/reader.py.

• file was removed as a function, so I replaced it with open.

• Since python 3 doesn't have classic-style classes, I removed all the instance code I could find. This means that __initargs__ isn't used anymore.

• u'...' isn't supported, so it's been replaced with just '...'.

• Unicode encoding now is str to bytes and decoding is bytes to str, so it required a lot of changes to change the order of encoding and decoding.

• unichr was replaced with chr.

• print now requires parenthesis.

• open('foo', 'rb') now returns a stream object that produces bytes instead of strs, so that had to be specially handled.

• Automatic tuple extraction with things like def foo(a, b, (c, d)): ... is no longer supported.

• There is no longer total ordering of all types, so [None, 1].sort() is now a TypeError. So, I added a basic __lt__ to yaml.nodes.Node which might need more attention.

• exec now requires parenthesis. Also, it doesn't seem like the values that get added to locals() are actually added to the local scope. This might be a bug.

• StringIO has been moved to the io module.

• Catching exceptions is now written like except YAMLError as exc.

• If you catch and then re raise another exception you can write raise YAMLError from other_exception to save the history of the other exception.

• I left in support for !!python/long and !!python/unicode for backwards compatibility.

• I removed all the (object) from class definition.

• mapping.keys() now returns an iterator, so I wrapped all the cases I could find that require it returning a list.

• I moved the sorting of a mapping to after the items in it have been turned into nodes so that it's more sortable.

Finally, in case it wasn't clear in the patch, I've removed the following tests:

D      tests/data/construct-python-unicode-utf8.data
D      tests/data/construct-python-unicode-ascii.data
D      tests/data/construct-python-unicode-utf8.code
D      tests/data/construct-python-unicode-ascii.code

And added:

A      tests/data/construct-python-bytes-utf8.data
A      tests/data/construct-python-bytes-ascii.data
A      tests/data/construct-python-bytes-utf8.code
A      tests/data/construct-python-bytes-ascii.code

Reported by rainwoodman@…, 4 years ago.

Please consider integrating libyaml with pkgconfig.

Reported by jstroud@…, 6 years ago.

Path resolving does not work for Nodes within Sequence Nodes

[This is current for pyyaml 3.04.]

For example:

yaml.add_path_resolver(u'!MyObject', (u'myobjs', [list, False]))

should match all of the items in the myobjs sequence in this yaml:

"""
--- !MyConfig
myobjs :
    - name: x
      root: z
    - name: p
      root: q
...
"""

The problem is here in resolver.py (line 210):

elif isinstance(index_check, int):
    if index_check != current_index:
         return

If I'm infering the intent of the code correctly

(u'myobjs', (list, False))

should match all elements of the sequence keyed by "myobjs". However,

isinstance(False, int)

always returns True because bool is a subclass of int, so index_check will only equal current_index for the first element when False is specified as the index_check.

Additionally, I believe another bug exists in resolver.py. According to the code in add_path_resolver(), index_check defaults to True when not specified (resolver.py, line 39):

if isinstance(element, (list, tuple)):
    if len(element) == 2:
        node_check, index_check = element
    elif len(element) == 1:
        node_check = element[0]
        index_check = True

The intendend meaning of True here is not clear because True causes every element in the sequence to fail the match. The relevant test is in check_resolver_prefix() (resolver.py, line 116):

if index_check is True and current_index is not None:
    return

Thus, any path not specifying an index_check will never match. Most insidious to the user is that this case is never reported nor an exception thrown. If index_check is True, this code will only continue to test if current_index is None, which should never happen for any node of any sequence, because any node of a sequence must, by virtue of its existence, have an index.

To fix these problems, I propose the following patch to resolver.py:

44c44
<                     index_check = True
---
>                     index_check = False
122c122
<         elif isinstance(index_check, int):
---
>         elif isinstance(index_check, int) and index_check is not False:

This will set the default value of index_check to match all elements of a sequence if no index_check is given and will not attempt to compare False to the value of index_check.

At the bare minimum, the following patch should be strongly considered if a good reason exists to leave the defalut of index_check to True.

122c122
<         elif isinstance(index_check, int):
---
>         elif isinstance(index_check, int) and index_check is not False:

Moreover, the code should provide some indication for what an index_check value of True actually means.

In fact, I propose re-writing the code to use None to match all elements, because

isinstance(None, int)

will always evaluate to False. This will also remove the question as to what the "other" bool value means because bools will not be used. I can provide a patch if this latter proposal sounds good to the developers.