iterator module

class iterator.Iterator(config: argparse.Namespace) → None

Bases: object

drawKey(is_train: bool) → typing.Tuple[int, int]

draw key

inputs: is_train
-is_train: a bool to flag to draw from train or test
outputs: tune_index, data_index
-tune_index: an index of tune -data_index: an index of data given tune index
evaluation() → typing.Tuple[typing.Dict[str, typing.List], typing.List, typing.List]

inputs: None

outputs: data_list, length_list, vocal_list

notes:
  • same return type like next, but affected by user instead random draw
getData() → typing.List[typing.List[typing.List[int]]]
getEpoch() → int
getSequence(input: int) → typing.List[int]
getStep() → int
get_user_intent() → typing.Tuple[int, int]

get user intent

inputs: None

outputs: tune_index, data_index
-tune_index: an index of tune -data_index: an index of data given tune index
notes:
  • user have to use chinses for keyword
mask(metaData: typing.Dict[str, typing.List]) → typing.List[typing.List[int]]

generate mask

inputs: meta_data
-data_data: a collection of data
outputs: mask_data
-mask_data: a mask for the result data
notes:
  • 1 if input data elment is non zero
next(is_train: bool = True) → typing.Tuple[typing.Dict[str, typing.List], typing.Dict[str, typing.List], typing.List]
inputs: is_train
  • is_train: boolean flag to decide to activate train

outputs: data_list, length_list, vocal_list

notes:
  • same return type like evaluation, but affected by user instead random draw
pad(metaData: typing.Dict[str, typing.List]) → typing.Dict[str, typing.List]

pad meta data

inputs: meta_data
-data_data: a collection of data
outputs: padded_meta_data
-padded_meta_data: a collection of padded data
padDecode(data: typing.List[typing.List[int]]) → typing.List[typing.List[int]]

pad decoder data

inputs: input_data
-input_data: a list of list with size [number of sentence], [length per sentence]
outputs: padded_decoder_data
-padded_decoder_data: a matrix with size [number of sentence, length of sentence]
notes:
  • for each pad, format is orginal sentence with GO at front
padEncode(data: typing.List[typing.List[int]]) → typing.List[typing.List[int]]

pad encoder data

inputs: input_data
-input_data: a list of list with size [number of sentence], [length per sentence]
outputs: padded_encoder_data
-padded_encoder_data: a matrix with size [number of sentence, length of sentence]
notes:
  • for each pad, format is reverse of orginal sentence with EOS at front
padResult(data: typing.List[typing.List[int]]) → typing.List[typing.List[int]]

pad result data

inputs: input_data
-input_data: a list of list with size [number of sentence], [length per sentence]
outputs: padded_result_data
-padded_result_data: a matrix with size [number of sentence, length of sentence]
notes:
  • for each pad, format is orginal sentence with EOS at end
padSentence(lenArray: typing.List[int]) → typing.Tuple[typing.List[numpy.ndarray], typing.List[numpy.ndarray], typing.List[int]]
pad sentence to [B,max(C)] then to [max(c),B] both for forward and backward sentence seq,
as well as a flatten 1d list of valid index
Notations:
  • C: 1D list containing ci length sequence over the batch. (在一批宋词中每首词的诗长,句数)

— max(C): max ci length sequence over the batch. (在一’批’宋词中最大词长数(在一批最多句数的词)) - B: batch size. (一批宋词的数量(多少首))

out : Sentenceindex,B

padTone(data: typing.List[typing.List[int]]) → typing.List[typing.List[int]]

pad tone data

inputs: tone_data
-tone_data: a list of list with size [number of sentence], [length per sentence]
outputs: padded_tone_data
-padded_tone_data: a matrix with size [number of sentence, length of sentence]
notes:
  • for each pad, format is orginal sentence with no sound at end
padVowel(key: int, index: int) → typing.List[typing.List[int]]
setEpoch(epoch: int) → None
setStep(step: int) → None
setupTest() → None

set up test envirnoment

inputs: None

outputs: None

notes:
  • tune_list: list of tune name
  • data_list: list of data index for a given tune
setupTrain() → None

set up training envirnoment

inputs: None

outputs: None

notes:
  • tune_list: list of tune name
  • data_list: list of data index for a given tune