| Current File : //usr/lib64/python3.4/lib2to3/__pycache__/refactor.cpython-34.pyc |
�
ʰ�`�m � @ s� d Z d d l m Z d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
m Z d d l m
Z
m Z m Z d d l m Z d d l m Z m Z d d
l m Z d d l m Z d d
d � Z Gd d � d e � Z d d � Z d d � Z d d � Z d d � Z e j d d f k rpd d l! Z! e! j" Z# d d � Z$ d d � Z% n e" Z# e Z$ e Z% d d � Z& Gd d! � d! e � Z' Gd" d# � d# e( � Z) Gd$ d% � d% e � Z* Gd& d' � d' e) � Z+ d S)(z�Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
� )�with_statementz#Guido van Rossum <guido@python.org>N)�chain� )�driver�tokenize�token)� find_root)�pytree�pygram)� btm_utils)� btm_matcherTc C s� t | g g d g � } t j j | j � } g } xn t t j | � � D]W } | j d � rI | j d � rI | r� | d d � } n | j | d d � � qI qI W| S)zEReturn a sorted list of all available fix names in the given package.�*�fix_z.py� N� ���)
�
__import__�os�path�dirname�__file__�sorted�listdir�
startswith�endswith�append)Z fixer_pkgZ
remove_prefixZpkgZ fixer_dirZ fix_names�name� r �(/usr/lib64/python3.4/lib2to3/refactor.py�get_all_fix_names! s r c @ s e Z d Z d S)�
_EveryNodeN)�__name__�
__module__� __qualname__r r r r r . s r c C s� t | t j t j f � r= | j d k r3 t � n | j h St | t j � rn | j re t | j � St � n t | t j � r� t
� } x5 | j D]* } x! | D] } | j t | � � q� Wq� W| St d | � � d S)zf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s)
�
isinstancer Z NodePatternZ LeafPattern�typer ZNegatedPatternZcontent�_get_head_typesZWildcardPattern�set�update� Exception)Zpat�r�p�xr r r r& 2 s
r& c C s t j t � } g } x� | D]� } | j r� y t | j � } Wn t k
r_ | j | � Yq� XxU | D] } | | j | � qg Wq | j d k r� | | j j | � q | j | � q Wx: t t j
j j � t j
j
� D] } | | j | � q� Wt | � S)z^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N)� collections� defaultdict�list�patternr& r r Z _accept_typer r
�python_grammarZ
symbol2number�values�tokens�extend�dict)Z
fixer_listZ
head_nodesZevery�fixerZheadsZ node_typer r r �_get_headnode_dictN s"
r7 c s � f d d � t � d � D� S)zN
Return the fully qualified names for fixers in the package pkg_name.
c s g | ] } � d | � q S)�.r )�.0�fix_name)�pkg_namer r �
<listcomp>k s z+get_fixers_from_package.<locals>.<listcomp>F)r )r; r )r; r �get_fixers_from_packageg s r= c C s | S)Nr )�objr r r � _identityn s r? r c C s | j d d � S)Nz
�
)�replace)�inputr r r �_from_system_newlinesu s rC c C s* t j d k r" | j d t j � S| Sd S)Nr@ )r �lineseprA )rB r r r �_to_system_newlinesw s rE c s� d } t j t j | � j � � � f d d � } t t j t j t j h � } t
� } yex^| � \ } } | | k r� q` q` | t j k r� | r� Pn d } q` | t j k r�| d k r�| � \ } } | t j k s� | d k r� Pn | � \ } } | t j k s| d k rPn | � \ } } | t j
k rV| d k rV| � \ } } n xa | t j k r�| j | � | � \ } } | t j
k s�| d k r�Pn | � \ } } qYWq` Pq` WWn t k
r�Yn Xt | � S)
NFc s t � � } | d | d f S)Nr r )�next)�tok)�genr r �advance� s z(_detect_future_features.<locals>.advanceT�from�
__future__�import�(�,)r �generate_tokens�io�StringIO�readline� frozensetr �NEWLINE�NL�COMMENTr' �STRING�NAME�OP�add�
StopIteration)�sourceZhave_docstringrI �ignore�featuresZtp�valuer )rH r �_detect_future_features� sD
r` c @ s e Z d Z d Z d S)�
FixerErrorz A fixer could not be loaded.N)r! r" r# �__doc__r r r r ra � s ra c @ sk e Z d Z i d d 6d d 6Z d Z d Z d d d d � Z d d
� Z d d � Z d
d � Z d d � Z
d d � Z d d d d � Z d d d d � Z
d d � Z d d d d � Z d d � Z d d d � Z d d � Z d! d" � Z d d d d# d$ � Z d d% d& � Z d' Z d( Z d) d* � Z d+ d, � Z d- d. � Z d/ d0 � Z d1 d2 � Z d3 d4 � Z d S)5�RefactoringToolF�print_function�write_unchanged_filesZFixr Nc C s� | | _ | p g | _ | j j � | _ | d k rI | j j | � n | j d re t j | _ n t j | _ | j j
d � | _ g | _ t
j d � | _ g | _ d | _ t j | j d t j d | j �| _ | j � \ | _ | _ g | _ t j � | _ g | _ g | _ x} t | j | j � D]f } | j rT| j j! | � q2| | j k rv| j j" | � q2| | j k r2| j j" | � q2q2Wt# | j � | _$ t# | j � | _% d S)z�Initializer.
Args:
fixer_names: a list of fixers to import
options: an dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Nrd re rc F�convert�logger)&�fixers�explicit�_default_options�copy�optionsr( r
�!python_grammar_no_print_statement�grammarr1 �getre �errors�loggingZ getLoggerrg � fixer_log�wroter ZDriverr rf �
get_fixers� pre_order�
post_order�files�bmZ
BottomMatcher�BMZ
bmi_pre_orderZbmi_post_orderr Z
BM_compatibleZ add_fixerr r7 �bmi_pre_order_heads�bmi_post_order_heads)�selfZ fixer_namesrl ri r6 r r r �__init__� s<
zRefactoringTool.__init__c C s� g } g } x�| j D]�} t | i i d g � } | j d d � d } | j | j � rx | t | j � d � } n | j d � } | j d j d d � | D� � } y t | | � } Wn( t
k
r� t d | | f � � Yn X| | j | j
� } | j r:| j d
k r:| | j k r:| j d | � q n | j d | � | j d
k ri| j | � q | j d k r�| j | � q t d | j � � q Wt j d � }
| j d |
� | j d |
� | | f S)a Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
r
r8 r N�_� c S s g | ] } | j � � q Sr )�title)r9 r+ r r r r< � s z.RefactoringTool.get_fixers.<locals>.<listcomp>zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %sZpreZpostzIllegal fixer order: %rZ run_order�key���)rh r �rsplitr � FILE_PREFIX�len�split� CLASS_PREFIX�join�getattr�AttributeErrorra rl rr ri � log_message� log_debugZorderr �operator�
attrgetter�sort) r| Zpre_order_fixersZpost_order_fixersZ fix_mod_path�modr: �partsZ
class_nameZ fix_classr6 Zkey_funcr r r rt � s8 #
zRefactoringTool.get_fixersc O s � d S)z Called when an error occurs.Nr )r| �msg�args�kwdsr r r � log_error s zRefactoringTool.log_errorc G s' | r | | } n | j j | � d S)zHook to log a message.N)rg �info)r| r� r� r r r r� s
zRefactoringTool.log_messagec G s' | r | | } n | j j | � d S)N)rg �debug)r| r� r� r r r r� s
zRefactoringTool.log_debugc C s d S)zTCalled with the old version, new version, and filename of a
refactored file.Nr )r| �old_text�new_text�filename�equalr r r � print_output! s z RefactoringTool.print_outputc C sP xI | D]A } t j j | � r5 | j | | | � q | j | | | � q Wd S)z)Refactor a list of files and directories.N)r r �isdir� refactor_dir�
refactor_file)r| �items�write�
doctests_onlyZ dir_or_filer r r �refactor& s
zRefactoringTool.refactorc
C s� t j d } x� t j | � D]� \ } } } | j d | � | j � | j � xe | D]] } | j d � rW t j j | � d | k rW t j j | | � } | j | | | � qW qW Wd d � | D� | d d � <q Wd S)z�Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
�pyzDescending into %sr8 r c S s% g | ] } | j d � s | � q S)r8 )r )r9 Zdnr r r r< A s z0RefactoringTool.refactor_dir.<locals>.<listcomp>N)
r �extsep�walkr� r� r r �splitextr� r� )
r| Zdir_namer� r� Zpy_ext�dirpathZdirnames� filenamesr �fullnamer r r r� / s
z RefactoringTool.refactor_dirc C s� y t | d � } Wn< t k
rQ } z | j d | | � d SWYd d } ~ Xn Xz t j | j � d } Wd | j � Xt | d d | �� } t | j � � | f SWd QXd S)zG
Do our best to decode a Python source file correctly.
�rbzCan't open %s: %sNr r* �encoding)NN)
�open�OSErrorr� r �detect_encodingrR �close�_open_with_encodingrC �read)r| r� �f�errr� r r r �_read_python_sourceC s z#RefactoringTool._read_python_sourcec C s | j | � \ } } | d k r% d S| d 7} | r� | j d | � | j | | � } | j sl | | k r� | j | | | | | � q| j d | � ni | j | | � } | j s� | r� | j r� | j t | � d d � | d | d | �n | j d | � d S)
zRefactors a file.Nr@ zRefactoring doctests in %szNo doctest changes in %sr r� r� zNo changes in %sr� )r� r� �refactor_docstringre �processed_file�refactor_string� was_changed�str)r| r� r� r� rB r� �output�treer r r r� S s
z RefactoringTool.refactor_filec C s� t | � } d | k r* t j | j _ n zb y | j j | � } WnE t k
r� } z% | j d | | j j | � d SWYd d } ~ Xn XWd | j | j _ X| | _
| j d | � | j | | � | S)aF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
rd zCan't parse %s: %s: %sNzRefactoring %s)
r` r
rm r rn Z parse_stringr) r� � __class__r! �future_featuresr� �
refactor_tree)r| �datar r^ r� r� r r r r� j s z RefactoringTool.refactor_stringc C s� t j j � } | ro | j d � | j | d � } | j sI | | k r_ | j | d | � q� | j d � nS | j | d � } | j s� | r� | j r� | j t | � d | � n
| j d � d S)Nz Refactoring doctests in stdinz<stdin>zNo doctest changes in stdinzNo changes in stdin)
�sys�stdinr� r� r� re r� r� r� r� )r| r� rB r� r� r r r �refactor_stdin� s
z RefactoringTool.refactor_stdinc
C s� x- t | j | j � D] } | j | | � q W| j | j | j � � | j | j | j � � | j j | j � � } x�t
| j � � rdx�| j j D]�} | | k r� | | r� | | j
d t j j d d � | j r� | | j
d t j j � n x\t | | � D]G} | | | k r9| | j | � n y t | � Wn t k
r^wYn X| j r}| | j k r}qn | j | � } | r| j | | � } | d k rV| j | � x9 | j � D]+ } | j s�g | _ n | j j | � q�W| j j | j � � } x? | D]4 } | | k r7g | | <n | | j | | � qWqVqqWq� q� Wq} Wx- t | j | j � D] } | j | | � q{W| j S)a� Refactors a parse tree (modifying the tree in place).
For compatible patterns the bottom matcher module is
used. Otherwise the tree is traversed node-to-node for
matches.
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
r� �reverseTN) r ru rv Z
start_tree� traverse_byrz r{ ry ZrunZleaves�anyr2 rh r� r ZBaseZdepthZkeep_line_orderZ
get_linenor/ �remover �
ValueErrorZfixers_applied�match� transformrA r r4 Z finish_treer� )
r| r� r r6 Z match_set�node�results�newZ new_matchesZfxrr r r r� � sJ
. z RefactoringTool.refactor_treec C s� | s
d Sxr | D]j } xa | | j D]R } | j | � } | r% | j | | � } | d k rw | j | � | } qw q% q% Wq Wd S)a Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
N)r% r� r� rA )r| rh Z traversalr� r6 r� r� r r r r� � s
zRefactoringTool.traverse_byc C s� | j j | � | d k rB | j | � d } | d k rB d Sn | | k } | j | | | | � | r� | j d | � | j s� d Sn | r� | j | | | | � n | j d | � d S)zR
Called when a file has been refactored and there may be changes.
Nr zNo changes to %szNot writing changes to %s)rw r r� r� r� re �
write_file)r| r� r� r� r� r� r� r r r r� � s z RefactoringTool.processed_filec % C s� y t | d d | �} Wn< t k
rW } z | j d | | � d SWYd d } ~ Xn XzV y | j t | � � Wn8 t k
r� } z | j d | | � WYd d } ~ Xn XWd | j � X| j d | � d | _ d S)z�Writes a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
�wr� zCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)r� r� r� r� rE r� r� rs )r| r� r� r� r� r� r� r r r r� s * zRefactoringTool.write_filez>>> z... c
C s� g } d } d } d } d } x4| j d d � D] } | d 7} | j � j | j � r� | d k r� | j | j | | | | � � n | } | g } | j | j � } | d | � } q1 | d k r
| j | | j � s� | | | j j � d k r
| j | � q1 | d k r8| j | j | | | | � � n d } d } | j | � q1 W| d k r�| j | j | | | | � � n d j
| � S)a� Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
Nr �keependsTr r@ r ) �
splitlines�lstripr �PS1r4 �refactor_doctest�find�PS2�rstripr r� )
r| rB r� �result�blockZ block_lineno�indent�lineno�line�ir r r r� ( s:
z"RefactoringTool.refactor_docstringc
s� y � j | | � � } Wn� t k
r� } zj � j j t j � rp x* | D] } � j d | j d � � qJ Wn � j d | | | j j
| � | SWYd d } ~ Xn X� j | | � r�t | � j
d d � } | d | d � | | d d � } } | d g | d k s t | � � | d j d � s@| d d 7<n � � j | j d � g } | r�| � � f d d
� | D� 7} q�n | S)
z�Refactors one doctest.
A doctest is given as a block of lines, the first of which starts
with ">>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
z
Source: %sr@ z+Can't parse docstring in %s line %s: %s: %sNr� Tr r c s! g | ] } � � j | � q Sr )r� )r9 r� )r� r| r r r< m s z4RefactoringTool.refactor_doctest.<locals>.<listcomp>r� r� )� parse_blockr) rg Z isEnabledForrq �DEBUGr� r� r� r� r! r� r� r� �AssertionErrorr r� �pop)
r| r� r� r� r� r� r� r� r� Zclippedr )r� r| r r� S s&
)# &z RefactoringTool.refactor_doctestc C s | j r d } n d } | j s4 | j d | � n1 | j d | � x | j D] } | j | � qN W| j r� | j d � x! | j D] } | j | � q� Wn | j rt | j � d k r� | j d � n | j d t | j � � x0 | j D]" \ } } } | j | | | � q� Wn d S) N�werez
need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:r zThere was 1 error:zThere were %d errors:)rs rw r� rr rp r� )r| r� �file�messager� r� r� r r r � summarizep s$
zRefactoringTool.summarizec C s1 | j j | j | | | � � } t � | _ | S)z�Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
)r Z parse_tokens� wrap_toksrS r� )r| r� r� r� r� r r r r� � s ! zRefactoringTool.parse_blockc c s� t j | j | | � j � } xe | D]] \ } } \ } } \ } }
} | | d 7} | | d 7} | | | | f | |
f | f Vq% Wd S)z;Wraps a tokenize stream to systematically modify start/end.r N)r rO � gen_lines�__next__) r| r� r� r� r3 r% r_ Zline0Zcol0Zline1Zcol1Z line_textr r r r� � s
(zRefactoringTool.wrap_toksc c s� | | j } | | j } | } xo | D]g } | j | � rT | t | � d � Vn4 | | j � d k rr d Vn t d | | f � � | } q' Wx d Vq� Wd S)z�Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
Nr@ zline=%r, prefix=%rr )r� r� r r� r� r� )r| r� r� Zprefix1Zprefix2�prefixr� r r r r� � s
zRefactoringTool.gen_lines) r! r" r# rj r� r� r} rt r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r r r rc � s:
4 ( O +
rc c @ s e Z d Z d S)�MultiprocessingUnsupportedN)r! r" r# r r r r r� � s r� c sa e Z d Z � f d d � Z d d d � f d d � Z � f d d � Z � f d d
� Z � S) �MultiprocessRefactoringToolc s/ t t | � j | | � d | _ d | _ d S)N)�superr� r} �queue� output_lock)r| r� �kwargs)r� r r r} � s z$MultiprocessRefactoringTool.__init__Fr c s[ | d k r( t t � � j | | | � Sy d d l � Wn t k
rR t � Yn X� j d k rq t d � � n � j � � _ � j � � _
� � f d d � t | � D� } z; x | D] } | j � q� Wt t � � j | | | � Wd � j j
� x$ t | � D] } � j j d � q Wx' | D] } | j � r*| j
� q*q*Wd � _ Xd S)Nr r z already doing multiple processesc s% g | ] } � j d � j � � q S)�target)ZProcess�_child)r9 r� )�multiprocessingr| r r r< � s z8MultiprocessRefactoringTool.refactor.<locals>.<listcomp>)r� r� r� r� � ImportErrorr� r� � RuntimeErrorZ
JoinableQueueZLockr� �range�startr� �putZis_alive)r| r� r� r� Z
num_processesZ processesr+ r� )r� )r� r| r r� � s2
z$MultiprocessRefactoringTool.refactorc
so | j j � } xY | d k rj | \ } } z t t | � j | | � Wd | j j � X| j j � } q Wd S)N)r� ro r� r� r� Z task_done)r| Ztaskr� r� )r� r r r� � s z"MultiprocessRefactoringTool._childc sE | j d k r( | j j | | f � n t t | � j | | � Sd S)N)r� r� r� r� r� )r| r� r� )r� r r r� � s z)MultiprocessRefactoringTool.refactor_file)r! r" r# r} r� r� r� r r )r� r r� � s
r� ),rb rK r �
__author__r r� rq r� r- rP � itertoolsr Zpgen2r r r Z
fixer_utilr r r r
r Zbur rx r r) r r& r7 r= r? � version_info�codecsr� r� rC rE r` ra �objectrc r� r� r r r r �<module> sF
(� �