Instantiate ELMo model in keras from weights in .h

Question

The pre-trained ELMo models are provided at https://allennlp.org/elmo.

How would I use the files provided?

I think that I have to reconstruct the model from the json file, and then load the weights from .hdf5 file into the model. But json format doesn't seem to work for keras.models.model_from_json. I got the error: ValueError: Improper config format: ...

Answer 1

Using tensorflow_hub to load ELMo model, an example:

import tensorflow as tf
import tensorflow_hub as hub
from keras.layers import Lambda
from keras.models import Input
from keras import backend as K
sess = tf.Session()
K.set_session(sess)

batch_size = 64
max_len = 100
elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())

def ElmoEmbedding(x):
    return elmo_model(inputs={"tokens": tf.squeeze(tf.cast(x,tf.string)),
                              "sequence_len": tf.constant(batch_size*[max_len])},
                      signature="tokens",
                      as_dict=True)["elmo"]
input_x = Input(shape=(max_len,), dtype=tf.string)
embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 128))(input_x)

Answer 2

elmo = ElmoEmbedder(
options_file='https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway_5.5B/elmo_2x4096_512_2048cnn_2xhighway_5.5B_options.json',
weight_file='https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway_5.5B/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights.hdf5')

But first import:

from allennlp.commands.elmo import ElmoEmbedder

You can replace the links with actual files too. Hope it helps