I wrote this to, (A) practice writing a script with the “scp” command, and (B) upload my “code” directory to my docuwiki installation so that it could be web-browsable. This is my first “real” script that I'm putting 'into the wild' per se, so have some mercy if it doesn't work too well :)
Note also that it's completely un-cross-platform. It assumes you're running Linux both on the client and the server, although I think that with some modification it may be runnable from a Windows client. The main roadblock is that Windows doesn't have an “SCP” command, and although I believe Python can use a module to open an SCP connection, I wanted to use the built-in one in case the user has individual settings for ssh-agent or public-key authentication. I don't have a WinXP machine to test on, so if someone wants to do that, feel free.
Also, the script suffers from a number of limitations: All pages must be lowercase, which effectively breaks Java files if you use the filename from DocuWiki when saving them back to disk (they should still be viewable, of course). This limitation is actually from within DocuWiki itself. Also, the only code files supported are those for Python, Java, and C. I use mostly Python, some Java, and have just started C, which is why those 3 are supported. It should be relatively straightforward to add more; see the source code and add the appropriate extensions.
These issues mean it's probably more useful as a showcase of source code than a way to collaborate on them. There's always patch for that.
To use this script, customize the constants USER_AT_HOST, DIRECTORY_TO_UPLOAD, and DOCUWIKI_INSTALL_DIRECTORY.
Okay, here goes:
#!/usr/bin/python #Adam Gomaa 2006. Started on October 24th. # A version 0.2 of the DocuWikiUpload script # Everything except the remote directory structure working on November 3rd. """Mirrors, formats, and uploads a directory of source code files into a Docuwiki installation. This 0.2 has been modularized. Please note that there is no provision for passwords; the SCP command should prompt you if one is necessary. Also note that this will upload files with a .docuwiki extension which will _not_ show up in docuwiki until the extensions have been changed to .txt. This was done to avoid thorny issues with unintentionally deleting .txt files in the cleanUp() module. The extension renaming can be done with a simple shell script: for file in *.docuwiki ; do mv $file `echo $file | sed 's/\(.*\.\)txt/\1docuwiki/'` ; done Run this in the docuwiki data/pages/ directory. I've decided not to do the renaming from within this script, because that would involve opening a remote shell or downloading/renaming/uploading over a remote connection, both of which seem like overkill to me. Lastly, you'll notice that os.walk is called several times. This needs to be done because the first function changes the results of os.walk, so we need to make sure it is performed again. Copyright (C) 2006 Adam Gomaa GNU GPL version 2 or greater. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. """ import os USER_AT_HOST="email@example.com" DIRECTORY_TO_UPLOAD="/home/user/code/" DOCUWIKI_INSTALL_DIRECTORY="~/host.com/docuwiki/" #Copyright notice print "This is Free software licensed under the GNU GPL." print "See included source for copyright details." ###Functions def mirrorCodeDir(): def copyFiles(name,root): #Open and read the file... readFile=file(os.path.join(root,name),'r') readData=readFile.read() readFile.close() writeFile=file(os.path.join(root,name.lower())+".docuwiki",'w') #... then write to (file).docuwiki according to what kind of source #code file it is. #The weird concatenation is so that </c0de> doesn't stop #this script from showing correctly in docuwiki. if ".py" in name: writeFile.write("<co"+"de python>\n"+readData+"\n</co"+"de>") elif ".java" in name: writeFile.write("<co"+"d:e java>\n"+readData+"\n</co"+"de>") elif ".pl" in name: writeFile.write("<co"+"de perl>\n"+readData+"\n</co"+"de>") elif ".c" in name or ".h" in name: writeFile.write("<co"+"de c>\n"+readData+"\n</co"+"de>") #Additional source code file extensions can be included here. #be sure to exclude binary files, you don't want those to show up in #docuwiki. writeFile.close() walkReturn=os.walk(DIRECTORY_TO_UPLOAD) for root, dirs, files in walkReturn: for name in files: # This should probably use a regular expression. if ('.py' in name or '.java' in name or '.c' in name or ".h" in name or ".pl" in name) and (".docuwiki.txt" not in name and ".class" not in name and ".pyc" not in name and "~" not in name and ".bak" not in name): copyFiles(name,root) def makeRemoteDirStructure(): def generateList(): mkdirlist= walkReturn=os.walk(DIRECTORY_TO_UPLOAD) for root, dirs, files in walkReturn: for directory in dirs: mkdirlist.append(os.path.join(root,directory)[len( DIRECTORY_TO_UPLOAD):]) return mkdirlist generateList() #need to use os functions to create the directories. They need to be made #in a particular order, eg, dir/ needs to be made before dir/subdir/ def uploadFiles(): walkReturn= os.walk(DIRECTORY_TO_UPLOAD) filelist= for root, dirs, files in walkReturn: for name in files: try: if name[-9:]=='.docuwiki': filelist.append(os.path.join(root,name)) except IndexError: pass commandString='' for each in filelist: commandString=commandString+" "+each commandString="scp "+commandString+' '+USER_AT_HOST commandString=commandString+":"+DOCUWIKI_INSTALL_DIRECTORY+"data/pages/" os.system(commandString) def cleanUp(): walkReturn= os.walk(DIRECTORY_TO_UPLOAD) """Iterates through the code directory, checks for files ending in the .docuwiki.txt, and removes them. """ removedFileCount=0 for root, dirs, files in walkReturn: for name in files: if name[-9:]=='.docuwiki': removedFileCount+=1 os.remove(os.path.join(root,name)) print "Removed",removedFileCount,"files." def main(): userInput1=raw_input("Mirror "+DIRECTORY_TO_UPLOAD+'?\n') if 'y' in userInput1: print "Mirroring "+DIRECTORY_TO_UPLOAD+"...", mirrorCodeDir() print 'done' userInput2=raw_input("Upload files?\n") if 'y' in userInput2: print "Uploading files...", uploadFiles() print 'done' userInput3= raw_input("Delete .docuwiki files?\n") if 'y' in userInput3: print "Cleaning .docuwiki files..." cleanUp() #makeRemoteDirStructure() #Not yet working, disabled for now. ###Main if __name__=="__main__": main()