BertfornextContencePredition是否将logits转换为布尔值？

Question

我正在尝试使用下一个句子预测模型，它是在BERT中预先训练的

我在类TFBertForNextSequencePrediction中使用了这个示例。我理解，seq_relationship_分数返回的logits指出下一句话是否属于上一个上下文。我尝试使用keras的Softmax，但它没有返回布尔值

import tensorflow as tf
from transformers import BertTokenizer, TFBertForNextSentencePrediction

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = TFBertForNextSentencePrediction.from_pretrained('bert-base-uncased')

input_ids = tf.constant(tokenizer.encode(["My dog is called Tetley.", "My dog is my best friend"], add_special_tokens=True))[None, :]  # Batch size 1

outputs = model(input_ids)
seq_relationship_scores = outputs[0]

print(seq_relationship_scores)

import keras
print(keras.activations.softmax(seq_relationship_scores, axis=-1))

我如何测试两个句子是否与seq_relationship_分数具有相同的上下文

https://huggingface.co/transformers/_modules/transformers/modeling_tf_bert.html#TFBertForNextSentencePrediction This model is a tf.keras.Model <https://www.tensorflow.org/api_docs/python/tf/keras/Model>__ sub-class. Use it as a regular TF 2.0 Keras Model and refer to the TF 2.0 documentation for all matter related to general usage and behavior.

输出如下所示：

tf.Tensor([[ 4.642335  -3.4926772]], shape=(1, 2), dtype=float32)
Using TensorFlow backend.
tf.Tensor([[9.9970692e-01 2.9300974e-04]], shape=(1, 2), dtype=float32)

Answer 1

反转逻辑以获取概率。然后将概率倒序排列，得到最接近的句子。您不需要在这里使用softmax，因为您正在查看相似性