Modify

Ticket #70 (closed defect: worksforme)

Opened 6 years ago

Last modified 43 hours ago

really slow on large input ...

Reported by: lal.george@… Owned by: xi
Priority: normal Component: pyyaml
Severity: normal Keywords:
Cc:

Description

yaml.load appears to be really (really) slow on large input. My string is

--- seqno : 1 outof : 1 result :

  • <1.6Mbyte string>

Both the input and output are fairly simple. Why does it take a long time to parse?

Lal

Attachments

Change History

comment:1 Changed 6 years ago by anonymous

is this with libyaml?

comment:2 Changed 6 years ago by xi

  • Status changed from new to closed
  • Resolution set to worksforme

Have you tried to parse the file with a libyaml based parser?

import yaml
yaml.load(input, Loader=yaml.CLoader)

comment:3 Changed 6 years ago by anonymous

Indeed, someone kindly suggested this. However, I am having trouble installing with libyaml.

I managed to build LibYaml? and Pyrex successfully, however when I try and install yaml with libyaml using:

python setup_with_libyaml.py install

I get the errors shown below.

It's complaining about no yaml.h file which is correct, and also a syntax error at the lines:

71 struct pyx_obj_5_yaml_CParser { 72 PyObject_HEAD 73 struct pyx_vtabstruct_5_yaml_CParser *pyx_vtab; 74 yaml_parser_t parser;

Any thoughts. My version of gcc is 3.2.3!

Any help is appreciated.

Best,

Lal

comment:4 Changed 6 years ago by xi

Could you attach the pyrex-generated C source and post the complete output of the compiler?

comment:5 Changed 6 years ago by anonymous

xi,

I finally managed to build with LibYaml?, but it was a real pain ... The primary issues had to do with the options to gcc for the include path to yaml.h, and the library path to libyaml.so.

I haven't include the generated C file, or the compiler output, as the C file was OK.

Included below is what I did eventually. If the correct method for installation is explained somewhere, then I must have missed it; however, what I went through should not have been necessary.

Best,

Lal


Step 1: Command:

python setup_with_libyaml.py install --prefix <path>

Failed on:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/lageorge/Tools/include/python2.5 -c ext/_yaml.c -o build/temp.linux-i686-2.5/ext/_yaml.o

In file included from ext/_yaml.c:26: ext/_yaml.h:2:18: yaml.h: No such file or directory

Fix:

redid gcc command with manual -I option added for yaml.h

Result:

Compiled with warnings.

ext/_yaml.c: In function `pyx_f_5_yaml_7CParser_init': ext/_yaml.c:461: warning: passing arg 2 of `yaml_parser_set_input' from incompatible pointer type ext/_yaml.c: In function `pyx_f_5_yaml_8CEmitter_init': ext/_yaml.c:4427: warning: passing arg 2 of `yaml_emitter_set_output' from incompatible pointer

Step 2) Command:

python setup_with_libyaml.py install --prefix <path>

Failed on:

gcc -pthread -shared build/temp.linux-i686-2.5/ext/_yaml.o -lyaml -o build/lib.linux-i686-2.5/_yaml.so

/usr/bin/ld: cannot find -lyaml collect2: ld returned 1 exit

Fix:

manually executed gcc with -L option to gcc. Defining environment variable LD_LIBRARY_PATH did not help

Result:

gcc ran to completion.

Step 3) Command:

python setup_with_libyaml.py install --prefix <path>

Result:

went to completion.

comment:6 Changed 43 hours ago by maskodok <galihadiputro87@…>

The only thing more I could hope for is documentation of all these features (other than reading through the code).  Cipto Junaedy Is this in process? Can I help? About  Unit Link Terbaik di Indonesia Commonwealth Life Investra Link

View

Add a comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.