How should I perform this file/ folder distributio

2019-02-24 22:47发布

站内文章 / 前端开发
27 0 0

问题:

I need to perform the following outlined task and am unsure how to proceed. Using Windows Server 2003, can I do this in a command script, or maybe a script task in SSIS? I have been using SSIS 2005 and know there is a file system task, but I've never used it before.

  • I have a file 'target.file' at local.
  • There is a 'target' folder on network server.
  • There are several hundred folders under 'target'.
  • There are 'backup' folders under some of those folders.
  • I need to copy 'target.file' to those folders under the 'target' folder.
    • But only copy/replace if there is already a 'target.file' existing.
    • If the 'target.file' exists, copy and replace the file to the backup folder if the backup folder exists.
    • If not, create the backup folder first.

回答1:

Sample package written in SSIS 2012 using C# and VB.NET

Here is a sample package written in SSIS 2012 that does what you are looking for using a script task. You don't need to use SSIS. You can even do this with a simple C# or VB.NET console application, but SSIS gives the flexibility to log information and schedule the jobs.

Folder structure (initial)

Let's assume that the folders are structured as shown below:

There is a source file that you would like to copy.

Source
    |- Sample_File.txt

Here is the target folder structure.

Target
    |- Target_1
    |    |- Archive
    |    |- Sample_File.txt
    |- Target_2
    |- Target_3
        |- Sample_File.txt

Create an SSIS package and create the folder variables:

Variable name       Data type  Value
------------------ ---------- ----------------------
Backup_FolderName  String      Archive
Source_FileName    String      Sample_File.txt
Source_FilePath    String
Source_Folder      String      D:\SSIS\Files\Source\
Target_Folder      String      D:\SSIS\Files\Target\

Select the variable Source_FilePath and click F4 to view the properties. Change the property EvaluateAsExpression to true. Click the ellipsis button next to the Expression property to open the Expression Builder. Set the expression to @[User::Source_Folder] + "\\" + @[User::Source_FileName].

You could have just one variable to store the source file path. I usually prefer to keep the source folder and the file name separate.

Drag and drop a script task onto the control flow tab. Double-click the script task to open the script task editor. On the script tab page, click the ellipsis button next to ReadOnlyVariables and select the following variables, because we will use these variables in the script task code.

User::Source_FilePath
User::Target_Folder
User::Backup_FolderName

Click the Edit Script... button and enter the code as shown below.

Script Task code in C# only for SSIS 2008 and above:

The script task code does the following:

  • It will check if the source file path is valid or not. If invalid, it will throw a message and quit the process.

  • It will check if the target folder is valid or not. If invalid, it will throw a message and quit the process.

  • If source file path and target folder are valid, the logic will loop through all the matching locations of the source file name in the sub-folders within target folder. If there are matching files, it will copy the target file to backup folder and then will overwrite the target file with source file.

  • The script task will emit the appropriate information so you can track the status within the progress/execution results tab on SQL Server Data Tools (SSDT) in SSIS 2012 or Business Intelligence Development Studio (BIDS) in SSIS 2005 - SSIS 2008 R2.

    region Namespaces

    using System; using System.Data; using Microsoft.SqlServer.Dts.Runtime; using System.Windows.Forms; using System.IO;

    endregion

    namespace ST_523853dfbc0d4123be43383671f8a6c6 { [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute] public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase { public void Main() { try { bool fireAgain = false; string backupFolder = string.Empty; string backupFilePath = string.Empty;

                string sourcFilePath = Dts.Variables["User::Source_FilePath"].Value.ToString();
            string targetFolder = Dts.Variables["User::Target_Folder"].Value.ToString();
            string backupFolderName = Dts.Variables["User::Backup_FolderName"].Value.ToString();

            if (String.IsNullOrEmpty(sourcFilePath) || !File.Exists(sourcFilePath))
            {
                // Check if a valid source file path was specified on the package variable
                Dts.Events.FireError(101, "Source path error", String.Format("You need to set a valid source file path in the package variable 'Source_FilePath'. Invalid path: '{0}'", sourcFilePath), string.Empty, 0);
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
            else if (String.IsNullOrEmpty(targetFolder) || !Directory.Exists(targetFolder))
            {
                // Check if a valid target folder was specified on the package variable
                Dts.Events.FireError(102, "Target folder error", String.Format("You need to set a valid target folder location in the package variable 'Target_Folder'. Invalid folder: '{0}'", targetFolder), string.Empty, 0);
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
            else
            {
                FileInfo sourceInfo = new FileInfo(sourcFilePath);

                // Loop through each file that matches the name of the source file
                foreach (string targetFilePath in Directory.EnumerateFiles(targetFolder, sourceInfo.Name, SearchOption.AllDirectories))
                {
                    FileInfo targetInfo = new FileInfo(targetFilePath);
                    backupFolder = Path.Combine(targetInfo.Directory.FullName, backupFolderName);
                    backupFilePath = Path.Combine(backupFolder, backupFolderName);

                    // If the backup folder does not exist in the folder within root target folder, create the backup folder.
                    if (!Directory.Exists(backupFolder))
                    {
                        Directory.CreateDirectory(backupFolder);
                        Dts.Events.FireInformation(401, "Backup folder created", String.Format("Backup folder '{0}' was created.", backupFolder), string.Empty, 0, ref fireAgain);
                    }

                    // Archive the target file to the backup folder.
                    File.Copy(targetFilePath, backupFilePath, true);
                    Dts.Events.FireInformation(402, "Target file archived", String.Format("Target file '{0}' was archived to the backup folder '{1}'.", targetFilePath, backupFolder), string.Empty, 0, ref fireAgain);

                    // Overwrite the target file with the source file.
                    File.Copy(sourcFilePath, targetFilePath, true);
                    Dts.Events.FireInformation(403, "Target file overwritten", String.Format("Target file '{0}' was overwritten with the source file '{1}'.", sourcFilePath, targetFilePath), string.Empty, 0, ref fireAgain);
                }

                Dts.TaskResult = (int)ScriptResults.Success;
            }
        }
        catch (Exception ex)
        {
            Dts.Events.FireError(100, "Unhandled exception", ex.ToString(), string.Empty, 0);
        }
    }

    #region ScriptResults declaration
    enum ScriptResults
    {
        Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
        Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
    };
    #endregion
}

    }

Script Task code in VB.NET for SSIS 2005 and above:

#Region "Imports"
Imports System
Imports System.Data
Imports System.Math
Imports System.IO
Imports Microsoft.SqlServer.Dts.Runtime
#End Region

<Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute()> _
<System.CLSCompliantAttribute(False)> _
Partial Public Class ScriptMain
    Inherits Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase

    Public Sub Main()

        Try
            Dim fireAgain As Boolean = False
            Dim backupFolder As String = String.Empty
            Dim backupFilePath As String = String.Empty

            Dim sourcFilePath As String = Dts.Variables("User::Source_FilePath").Value.ToString()
            Dim targetFolder As String = Dts.Variables("User::Target_Folder").Value.ToString()
            Dim backupFolderName As String = Dts.Variables("User::Backup_FolderName").Value.ToString()

            If String.IsNullOrEmpty(sourcFilePath) OrElse Not File.Exists(sourcFilePath) Then
                ' Check if a valid source file path was specified on the package variable
                Dts.Events.FireError(101, "Source path error", String.Format("You need to set a valid source file path in the package variable 'Source_FilePath'. Invalid path: '{0}'", sourcFilePath), String.Empty, 0)
                Dts.TaskResult = ScriptResults.Failure

            ElseIf String.IsNullOrEmpty(targetFolder) OrElse Not Directory.Exists(targetFolder) Then
                ' Check if a valid target folder was specified on the package variable
                Dts.Events.FireError(102, "Target folder error", String.Format("You need to set a valid target folder location in the package variable 'Target_Folder'. Invalid folder: '{0}'", targetFolder), String.Empty, 0)
                Dts.TaskResult = ScriptResults.Failure

            Else
                Dim sourceInfo As FileInfo = New FileInfo(sourcFilePath)

                ' Loop through each file that matches the name of the source file
                For Each targetFilePath As String In Directory.EnumerateFiles(targetFolder, sourceInfo.Name, SearchOption.AllDirectories)

                    Dim targetInfo As FileInfo = New FileInfo(targetFilePath)
                    backupFolder = Path.Combine(targetInfo.Directory.FullName, backupFolderName)
                    backupFilePath = Path.Combine(backupFolder, backupFolderName)

                    ' If the backup folder does not exist in the folder within root target folder, create the backup folder.
                    If Not Directory.Exists(backupFolder) Then
                        Directory.CreateDirectory(backupFolder)
                        Dts.Events.FireInformation(401, "Backup folder created", String.Format("Backup folder '{0}' was created.", backupFolder), String.Empty, 0, fireAgain)
                    End If

                    ' Archive the target file to the backup folder.
                    File.Copy(targetFilePath, backupFilePath, True)
                    Dts.Events.FireInformation(402, "Target file archived", String.Format("Target file '{0}' was archived to the backup folder '{1}'.", targetFilePath, backupFolder), String.Empty, 0, fireAgain)

                    ' Overwrite the target file with the source file.
                    File.Copy(sourcFilePath, targetFilePath, True)
                    Dts.Events.FireInformation(403, "Target file overwritten", String.Format("Target file '{0}' was overwritten with the source file '{1}'.", sourcFilePath, targetFilePath), String.Empty, 0, fireAgain)

                Next

                Dts.TaskResult = ScriptResults.Success

            End If

        Catch ex As Exception
            Dts.Events.FireError(100, "Unhandled exception", ex.ToString(), String.Empty, 0)
            Dts.TaskResult = ScriptResults.Failure
        End Try

    End Sub

#Region "ScriptResults declaration"
    Enum ScriptResults
        Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success
        Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
    End Enum

#End Region

End Class

When an invalid source file path is provided, the package will throw the below error message:

When an invalid target folder is provided, the package will throw the below error message:

When source and target locations are valid, the package will execute successfully. In this example,

  • there was a backup folder under Target_1, so no folder was created but file was copied to backup folder.
  • There was no matching file in Target_2, so no action was taken.
  • Backup folder was created in Target_3, the file was copied to target location and then overwritten with source file.

Folder structure (Final)

The target location will look like as shown below after the package execution.

Target
    |- Target_1
    |    |- Archive
    |        |- Sample_File.txt
    |    |- Sample_File.txt
    |- Target_2
    |- Target_3
        |- Archive
            |- Sample_File.txt
        |- Sample_File.txt


回答2:

Apparently @Siva's done tremendous work answering your question. I wonder, however, if the following would do the job much worse (even though it most probably didn't take me as long to come up with this script):

@ECHO OFF

SET "srcname=target.file"
SET "srcpath=D:\path\to\source"
SET "tgtpath=\\remotehost\shared\folder"
SET "bakname=Backup"

FOR /D %%D IN ("%tgtpath%\*") DO (
  IF EXIST "%%D\%srcname%" (
    IF NOT EXIST "%%D\%bakname%" MKDIR "%%D\%bakname%"
    COPY /Y "%%D\%srcname%" "%%D\%bakname%"
    COPY /Y "%srcpath%\%srcname%" "%%D"
  )
)


标签： batch-file ssis
举报

收藏的人(0)

Ta的文章 更多文章
登录 后发表评论
0条评论
还没有人评论过~