| Current File : //lib/python3.6/site-packages/bs4/__pycache__/diagnose.cpython-36.pyc |
3
#6]� � @ s d Z dZddlZddlmZ ddlmZ ddlZddlmZm Z ddl
m Z ddl Z ddl
Z
ddlZddlZddlZddlZddlZddlZdd � Zd#d d �ZG d
d� de�Zdd� ZdZdZd$dd�Zd%dd�Zd&dd�Zd'd d �Z d(d d!�Z e d"k�reej j � � dS ))z=Diagnostic functions, mainly for use when doing tech support.ZMIT� N)�StringIO)�
HTMLParser)�
BeautifulSoup� __version__)�builder_registryc ; C sX t dt � t dtj � dddg}x>|D ]6}x0tjD ]}||jkr6P q6W |j|� t d| � q*W d|kr�|jd� y*dd l m
} t d
d j t t
|j�� � W n* tk
r� } zt d � W Y d
d
}~X nX d|k�r ydd
l}t d|j � W n, tk
�r } zt d� W Y d
d
}~X nX t| d��r4| j� } n�| jd��sL| jd��rdt d| � t d� d
S y:tjj| ��r�t d| � t| ��}|j� } W d
Q R X W n tk
�r� Y nX t � x�|D ]�}t d| � d} yt| |d�}
d} W n8 tk
�r" } zt d| � tj � W Y d
d
}~X nX | �rBt d| � t |
j � � t d d � �q�W d
S ) z/Diagnostic suite for isolating common problems.z'Diagnostic running on Beautiful Soup %szPython version %sz html.parser�html5lib�lxmlz;I noticed that %s is not installed. Installing it may help.zlxml-xmlr )�etreezFound lxml version %s�.z.lxml is not installed or couldn't be imported.NzFound html5lib version %sz2html5lib is not installed or couldn't be imported.�readzhttp:zhttps:z<"%s" looks like a URL. Beautiful Soup is not an HTTP client.zpYou need to use some other library to get the document behind the URL, and feed that document to Beautiful Soup.z7"%s" looks like a filename. Reading data from the file.z#Trying to parse your markup with %sF)�featuresTz %s could not parse the markup.z#Here's what %s did with the markup:�-�P ) �printr �sys�versionr Zbuildersr �remove�appendr r �join�map�strZ LXML_VERSION� ImportErrorr �hasattrr �
startswith�os�path�exists�open�
ValueErrorr � Exception� traceback� print_excZprettify) �dataZ
basic_parsers�nameZbuilderr �er �fp�parser�success�soup� r) � /usr/lib/python3.6/diagnose.py�diagnose sj
r+ Tc K sN ddl m} x<|jt| �fd|i|��D ] \}}td||j|jf � q(W dS )z�Print out the lxml events that occur during parsing.
This lets you see how lxml parses a document when no Beautiful
Soup code is running.
r )r �htmlz %s, %4s, %sN)r r Z iterparser r �tag�text)r" r, �kwargsr Zevent�elementr) r) r* �
lxml_traceZ s $r1 c @ s` e Zd ZdZdd� Zdd� Zdd� Zdd � Zd
d � Zd d
� Z dd� Z
dd� Z dd� Z dd� Z
dS )�AnnouncingParserz?Announces HTMLParser parse events, without doing anything else.c C s t |� d S )N)r )�self�sr) r) r* �_pg s zAnnouncingParser._pc C s | j d| � d S )Nz%s START)r5 )r3 r# Zattrsr) r) r* �handle_starttagj s z AnnouncingParser.handle_starttagc C s | j d| � d S )Nz%s END)r5 )r3 r# r) r) r* �
handle_endtagm s z AnnouncingParser.handle_endtagc C s | j d| � d S )Nz%s DATA)r5 )r3 r" r) r) r* � handle_datap s z AnnouncingParser.handle_datac C s | j d| � d S )Nz
%s CHARREF)r5 )r3 r# r) r) r* �handle_charrefs s z AnnouncingParser.handle_charrefc C s | j d| � d S )Nz %s ENTITYREF)r5 )r3 r# r) r) r* �handle_entityrefv s z!AnnouncingParser.handle_entityrefc C s | j d| � d S )Nz
%s COMMENT)r5 )r3 r" r) r) r* �handle_commenty s z AnnouncingParser.handle_commentc C s | j d| � d S )Nz%s DECL)r5 )r3 r" r) r) r* � handle_decl| s z AnnouncingParser.handle_declc C s | j d| � d S )Nz%s UNKNOWN-DECL)r5 )r3 r" r) r) r* � unknown_decl s z AnnouncingParser.unknown_declc C s | j d| � d S )Nz%s PI)r5 )r3 r" r) r) r* � handle_pi� s zAnnouncingParser.handle_piN)�__name__�
__module__� __qualname__�__doc__r5 r6 r7 r8 r9 r: r; r< r= r> r) r) r) r* r2 d s r2 c C s t � }|j| � dS )z�Print out the HTMLParser events that occur during parsing.
This lets you see how HTMLParser parses a document when no
Beautiful Soup code is running.
N)r2 Zfeed)r" r& r) r) r* �htmlparser_trace� s rC ZaeiouZbcdfghjklmnpqrstvwxyz� c C s> d}x4t | �D ](}|d dkr$t}nt}|tj|�7 }qW |S )z#Generate a random word-like string.� � r )�range� _consonants�_vowels�random�choice)�lengthr4 �i�tr) r) r* �rword� s rO � c C s dj dd� t| �D ��S )z'Generate a random sentence-like string.� c s s | ]}t tjd d��V qdS )rP � N)rO rJ �randint)�.0rM r) r) r* � <genexpr>� s z rsentence.<locals>.<genexpr>)r rG )rL r) r) r* � rsentence� s rV �� c C s� dddddddg}g }x~t | �D ]r}tjdd �}|dkrRtj|�}|jd
| � q |d krr|jttjd d ��� q |d
kr tj|�}|jd| � q W ddj|� d S )z+Randomly generate an invalid HTML document.�pZdiv�spanrM �bZscript�tabler � z<%s>� rP rF z</%s>z<html>�
z</html>)rG rJ rS rK r rV r )� num_elementsZ tag_names�elementsrM rK Ztag_namer) r) r* �rdoc� s
ra 順 c
C s( t dt � t| �}t dt|� � x�dddgddgD ]z}d}y"tj� }t||�}tj� }d}W n6 tk
r� } zt d | � tj� W Y d
d
}~X nX |r6t d ||| f � q6W d d
l m
} tj� }|j |� tj� }t d|| � d d
l } | j
� }tj� }|j|� tj� }t d|| � d
S )z.Very basic head-to-head performance benchmark.z1Comparative parser benchmark on Beautiful Soup %sz3Generated a large invalid HTML document (%d bytes).r r, r z html.parserFTz %s could not parse the markup.Nz"BS4+%s parsed the markup in %.2fs.r )r z$Raw lxml parsed the markup in %.2fs.z(Raw html5lib parsed the markup in %.2fs.)r r ra �len�timer r r r! r r ZHTMLr r �parse)
r_ r" r& r' �ar( rZ r$ r r r) r) r* �benchmark_parsers� s4
rg r c C sX t j� }|j}t| �}tt||d�}tjd|||� tj |�}|j
d� |j dd� d S )N)�bs4r" r& z bs4.BeautifulSoup(data, parser)Z
cumulativez
_html5lib|bs4�2 ) �tempfileZNamedTemporaryFiler# ra �dictrh �cProfileZrunctx�pstatsZStatsZ
sort_statsZ print_stats)r_ r& Z
filehandle�filenamer" �varsZstatsr) r) r* �profile� s
rp �__main__)T)rD )rP )rW )rb )rb r )!rB Z __license__rl �ior Z html.parserr rh r r Z bs4.builderr r rm rJ rj rd r r r+ r1 r2 rC rI rH rO rV ra rg rp r? �stdinr r) r) r) r* �<module> s8 C
!