使用GNU / Linux系统调用`splice`实现零拷贝套接字在哈斯克尔插槽的数据传输(Usin

2019-06-23 11:43发布

更新:尼莫先生的回答有助于解决问题! 下面的代码包含修复! nb Falsenb True下面的电话。

还有一个新的Haskell包称为splice (,它具有最好的已知插座的特定OS和便携式实现套接字数据传送循环)。

我有以下的(哈斯克尔)代码:

#ifdef LINUX_SPLICE
#include <fcntl.h>
{-# LANGUAGE CPP #-}
{-# LANGUAGE ForeignFunctionInterface #-}
#endif

module Network.Socket.Splice (
    Length
  , zeroCopy
  , splice
#ifdef LINUX_SPLICE
  , c_splice
#endif
  ) where

import Data.Word
import Foreign.Ptr

import Network.Socket
import Control.Monad
import Control.Exception
import System.Posix.Types
import System.Posix.IO

#ifdef LINUX_SPLICE
import Data.Int
import Data.Bits
import Unsafe.Coerce
import Foreign.C.Types
import Foreign.C.Error
import System.Posix.Internals
#else
import System.IO
import Foreign.Marshal.Alloc
#endif


zeroCopy :: Bool
zeroCopy =
#ifdef LINUX_SPLICE
  True
#else
  False
#endif


type Length =
#ifdef LINUX_SPLICE
  (#type size_t)
#else
  Int
#endif


-- | The 'splice' function pipes data from
--   one socket to another in a loop.
--   On Linux this happens in kernel space with
--   zero copying between kernel and user spaces.
--   On other operating systems, a portable
--   implementation utilizes a user space buffer
--   allocated with 'mallocBytes'; 'hGetBufSome'
--   and 'hPut' are then used to avoid repeated 
--   tiny allocations as would happen with 'recv'
--   'sendAll' calls from the 'bytestring' package.
splice :: Length -> Socket -> Socket -> IO ()
splice l (MkSocket x _ _ _ _) (MkSocket y _ _ _ _) = do

  let e  = error "splice ended"

#ifdef LINUX_SPLICE

  (r,w) <- createPipe
  print ('+',r,w)
  let s  = Fd x -- source
  let t  = Fd y -- target
  let c  = throwErrnoIfMinus1 "Network.Socket.Splice.splice"
  let u  = unsafeCoerce :: (#type ssize_t) -> (#type size_t)
  let fs = sPLICE_F_MOVE .|. sPLICE_F_MORE
  let nb v = do setNonBlockingFD x v
                setNonBlockingFD y v
  nb False
  finally
    (forever $ do 
       b <- c $ c_splice s nullPtr w nullPtr    l  fs
       if b > 0
         then   c_splice r nullPtr t nullPtr (u b) fs)
         else   e
    (do closeFd r
        closeFd w
        nb True
        print ('-',r,w))

#else

  -- ..    

#endif


#ifdef LINUX_SPLICE
-- SPLICE

-- fcntl.h
-- ssize_t splice(
--   int          fd_in,
--   loff_t*      off_in,
--   int          fd_out,
--   loff_t*      off_out,
--   size_t       len,
--   unsigned int flags
-- );

foreign import ccall "splice"
  c_splice
  :: Fd
  -> Ptr (#type loff_t)
  -> Fd
  -> Ptr (#type loff_t)
  -> (#type size_t)
  -> Word
  -> IO (#type ssize_t)

sPLICE_F_MOVE :: Word
sPLICE_F_MOVE = (#const "SPLICE_F_MOVE")

sPLICE_F_MORE :: Word
sPLICE_F_MORE = (#const "SPLICE_F_MORE")
#endif

注: 上面的代码,现在只是工作! 下面是不再有效感谢尼莫!

我称splice如上具有两个开口和连接的套接字(其已经用于传输使用任一套接字API握手数据的最小量限定sendrecv呼叫或转换至把手和与用于hGetLinehPut )和I不断收到:

Network.Socket.Splice.splice: resource exhausted (Resource temporarily unavailable)

在第一c_splice调用点: c_splice返回-1 ,并设置了一些errno的值(可能EAGAIN )读取resource exhausted | resource temporarily unavailable resource exhausted | resource temporarily unavailable时抬头。

我测试呼叫splice不同的Length值: 10248192

Answer 1:

我不知道哈斯克尔,但“资源暂时不可用”是EAGAIN

它看起来像哈斯克尔将其套接字非阻塞模式在默认情况下。 所以,如果你试图从一个读取时没有数据,或者尝试写一个当其缓冲区已满,你会失败,并EAGAIN

弄清楚如何插座改变成阻塞模式,我敢打赌,你会解决你的问题。

[更新]

另外,调用selectpoll试图读取或写入套接字之前。 但你仍然需要处理EAGAIN ,因为有罕见的角落情况下,Linux的select将显示一个插槽中安装时,其实不然。



Answer 2:

请问sendfile()为你的系统调用的工作? 如果是这样你可以使用的sendfile包 。



文章来源: Using GNU/Linux system call `splice` for zero-copy Socket to Socket data transfers in Haskell