2024-06-28 15:34:20 发布
网友
我有两个收藏。一个由m1维度中的m1点组成,另一个由k维度中的m2点组成。我需要计算两个集合中每一对之间的成对距离。在
基本上有两个矩阵Am1,k和Bm2,k我需要得到一个矩阵Cm1,m2。在
我可以很容易地在scipy中使用distance.sdist并从许多距离度量中选择一个,我也可以在TF中循环执行此操作,但我甚至无法计算如何使用矩阵操作来实现这一点,即使对于欧几里德距离。在
对于任意维数的张量(即包含(…,N,d)向量),这将起到作用。注意,它不是在集合之间(例如,不像scipy.spatial.distance.cdist),而是在一批向量中(比如scipy.spatial.distance.pdist)
scipy.spatial.distance.cdist
scipy.spatial.distance.pdist
import tensorflow as tf import string def pdist(arr): """Pairwise Euclidean distances between vectors contained at the back of tensors. Uses expansion: (x - y)^T (x - y) = x^Tx - 2x^Ty + y^Ty :param arr: (..., N, d) tensor :returns: (..., N, N) tensor of pairwise distances between vectors in the second-to-last dim. :rtype: tf.Tensor """ shape = tuple(arr.get_shape().as_list()) rank_ = len(shape) N, d = shape[-2:] # Build a prefix from the array without the indices we'll use later. pref = string.ascii_lowercase[:rank_ - 2] # Outer product of points (..., N, N) xxT = tf.einsum('{0}ni,{0}mi->{0}nm'.format(pref), arr, arr) # Inner product of points. (..., N) xTx = tf.einsum('{0}ni,{0}ni->{0}n'.format(pref), arr, arr) # (..., N, N) inner products tiled. xTx_tile = tf.tile(xTx[..., None], (1,) * (rank_ - 1) + (N,)) # Build the permuter. (sigh, no tf.swapaxes yet) permute = list(range(rank_)) permute[-2], permute[-1] = permute[-1], permute[-2] # dists = (x^Tx - 2x^Ty + y^Tx)^(1/2). Note the axis swapping is necessary to 'pair' x^Tx and y^Ty return tf.sqrt(xTx_tile - 2 * xxT + tf.transpose(xTx_tile, permute))
几个小时后,我终于找到了如何在Tensorflow中实现这一点。我的解决方案只适用于欧几里德距离,而且相当冗长。我也没有数学证明(只是大量的手绘,我希望做得更严谨):
import tensorflow as tf import numpy as np from scipy.spatial.distance import cdist M1, M2, K = 3, 4, 2 # Scipy calculation a = np.random.rand(M1, K).astype(np.float32) b = np.random.rand(M2, K).astype(np.float32) print cdist(a, b, 'euclidean'), '\n' # TF calculation A = tf.Variable(a) B = tf.Variable(b) p1 = tf.matmul( tf.expand_dims(tf.reduce_sum(tf.square(A), 1), 1), tf.ones(shape=(1, M2)) ) p2 = tf.transpose(tf.matmul( tf.reshape(tf.reduce_sum(tf.square(B), 1), shape=[-1, 1]), tf.ones(shape=(M1, 1)), transpose_b=True )) res = tf.sqrt(tf.add(p1, p2) - 2 * tf.matmul(A, B, transpose_b=True)) with tf.Session() as sess: sess.run(tf.global_variables_initializer()) print sess.run(res)
对于任意维数的张量(即包含(…,N,d)向量),这将起到作用。注意,它不是在集合之间(例如,不像
scipy.spatial.distance.cdist
),而是在一批向量中(比如scipy.spatial.distance.pdist
)几个小时后,我终于找到了如何在Tensorflow中实现这一点。我的解决方案只适用于欧几里德距离,而且相当冗长。我也没有数学证明(只是大量的手绘,我希望做得更严谨):
相关问题 更多 >
编程相关推荐