large file transfer
Simple File Transfer 0.4.14
Simple File Transfer is a pure useless program which allows you exchange files with remote computers via Internet. more>>
It should run on every POSIX compliant Operating System, but I cant give you any complete list.
If you want to compile it under *BSD, maybe youll have to disable the uname() call by running the configure script with the --with-uname=no option.
Enhancements:
- Added the support of hostnames for the destination server
VFU File Manager 4.05
VFU is console (text mode) file manager for UNIX/Linux. more>>
During the years I tried to make short attractive list of features which VFU has. Attractive as Advertising above :) and it does not seem to work... at least it didnt for me and I never liked it.
I made all possible to make VFU cover all file managing needs and offer large set of behaviour options. If I succeeded or not, you can decide for yourself but without trying I believe noone can say for sure.
Installation:
1. how to compile vfu
run `make from vfu base directory
this should compile everything
to compile each part of vfu manually do this:
-- go to `vslib directory
-- run `make
-- go to `vfu directory
-- run `make
if something goes wrong, check these:
-- if your `curses.h file locations is not `/usr/include/ncurses
you have to change this in the Makefile.
-- if vslib library is not in the `../vslib directory you also
have to change this in the Makefile.
2. how to install vfu
run `install script from vfu base directory
install script checks if all required files are available/built and then does this:
cp vfu/vfu rx/rx_* /usr/local/bin
cp vfu.1 /usr/local/man/man1
cp vfu.conf /usr/local/etc
3. how to install vfu manually
-- you have to copy `vfu in the `/usr/local/bin or
`/usr/bin directory and set mode to 755 `rwxr-xr-x
the owner is not significant ( root is also possible ).
-- there is preliminary man page ( vfu.1 ) which could be
copied to /usr/man/man1.
-- copy all `rx/rx_* tools to /usr/local/bin
-- install Net::FTP perl module if needed.
(this is used for FTP support)
WARNING: make sure to remove all old personal cache files!
Upstream Log Transfer System 0.2.0
Upstream is a collection of tools that allow users to send in system specific log and troubleshooting data to support personnel. more>>
Upstream aims to simplify the support cycle and make it more efficient. One of the main problems that support staff face when helping people via IRC or mailing lists is getting user log files across.
Often this process is time consuming, and many new users may even be unable to find / copy these logs. Upstream does this for them by finding and transmitting the log files relevant to a specific issue.
Usage:
We hope to turn this,
< newuser >hi, something is wrong with my resolution, how can i fix it?
< support_person >newuser, what is your video card, and which driver is xorg using?
< newuser >err... how can i find out?
< support_person > cat /var/log/Xorg.0.log and pastebin it
< newuser >pastebin?
< support_person >http://pastebin.ca
< newuser >how do i copy? it wont copy!
In to this,
< newuser >hi, something is wrong with my resolution, how can i fix it?
< support_person >newuser, please run Canoe (System -> Help -> Canoe) and choose Video support. Give me the url you get once its done.
Apple Disk Transfer ProDOS 1.0.2
Apple Disk Transfer ProDOS transfers diskettes and logical disk images between Apple ][-era computers and the modern world. more>>
Main features:
- Compatibility with any device ProDOS can read
- Compatibility with any Apple ][ (or clone) computer with 64k memory
- Compatibility with many logical disk image formats: .DSK, .PO, .NIB, 2IMG
- Server compatibility with original ADT client program
- Server compatibility with Windows, Mac OSX, Linux, and probably Solaris
- Ability to bootstrap an Apple ][ from bare metal over serial or cassette ports
- Ability to send floppies in "batch" mode without having to name each one
Server
The server program runs on a computer capable of running Java. Depending on how you want to connect to your Apple, you might also need a serial port and cables, an Uthernet card for your apple, or a couple of audio patch cables. The server offers a compact user interface that shows what communications are taking place between the host and the Apple ][.
The servers primary role is to send and receive disk images as requested from the client. But if you have recently acquired an Apple and a disk drive, and you have no software for it - youre in a bit of a tricky situation if you want to move software from the Internet all the way to your shiny new Apple. ADTPros server can help get you get bootstrapped.
Client
The client side runs on the Apple ][. It handles most of the user interaction. When choosing disks/volumes to transfer, anything that ProDOS can see is fair game. Transferring data occurs with a 20k buffer on the Apple, so all transfers are broken up into 20k chunks. A progress indicator shows how far it is into the current chunk, as well as a running count of the total progress.
Enhancements:
- This release has been enhanced with Jean-Marc Boutillon (Deckard)s FASTDSK fast Disk II reading routines.
- This results in a speed boost of 25%-33% for Disk II to host transfers.
- Bootstrapping operations have been reduced, as there is no longer a dependency on ProDOS BASIC.
Fast Data Transfer 0.8.0
Fast Data Transfer is an application for efficient data transfers that is capable of reading and writing at disk speed. more>>
It can be used to stream a large set of files across the network, so a large dataset composed of thousands of files can be sent or received at full speed, without the network transfer restarting between files.
The project is written in Java, runs an all major platforms, and is easy to use.
Main features:
- Streams a dataset (list of files) continuously, using a managed pool of buffers through one or more TCP sockets.
- Uses independent threads to read and write on each physical device
- Transfers data in parallel on multiple TCP streams, when necessary
- Uses appropriate-sized buffers for disk I/O and for the network
- Restores the files from buffers asynchronously
- Resumes a file transfer session without loss, when needed
Audio Transcriber 0.9.2B
Audio Transcriber records large audio samples, like a tape or radio broadcast, to your hard drive and splits it into tracks. more>>
The goal of Audio Transcriber is to make it easy to transfer audio tapes, LPs, or radio broadcasts to tracks on CD-R/CD-RW.
GNOME Transfer Manager 0.4.12
GNOME Transfer Manager allows the user to retrieve multiple files from the web. more>>
These files can be retrieved in multiple parts and each part retrieved on a separate session that the user is connected to the Internet. This is most useful to users with dialup connections. The program performs these tasks using wget as its back-end.
The program supports CORBA. This makes it easy for other programs to use GTransferManager to handle the transfer of files from the Internet.
The program also has an applet which communicates with program using CORBA. The applet can launch the program, request for a new download or accept drops of URLs from netscape, gFTP, gmc and give these URLs to GTM.
Fast Secure File System 0.1.1
Fast Secure File System is a secure, distributed, scalable, user-space file system. more>>
File systems are easily the most evident, from the point of view of users, component of an operating system. Through file systems it is possible to organize data in a wide variety of ways, and access resources through a common interface.
Users can nowadays not only store and retrieve documents, but also find information on running processes and system settings (through ProcFS), access and manipulate e-mail (for example with GmailFS), or perform several other operations.
In several circumstances and scenarios it is desirable to protect stored files and directories from manipulation by unknown or malicious users: financial or health-related data, confidential documents, or any kind of personal or sensitive data may need to be stored securely, in such a way that it can not be examined or modified freely by third parties.
Most file systems do not take action in this sense, and external cryptographic utilities are sometimes employed to secure data before storage. While this can be a perfectly secure solution, it is not transparent to users.
Distributed file systems propose efficient ways of accessing data remotely as if it resided on the local machine; when it comes to dealing with securely stored data as in the examples above, care must be taken to preserve confidentiality and integrity also during network transfer.
Not all distributed file systems accomplish this task, weakening the overall security of the system, or do so inefficiently, making it inconvenient for users.
FSFS is a secure, distributed file system in users space, written in C with much help from FUSE and OpenSSL. It lets users store and retrieve data securely and transparently, knowing that it is protected both on permanent storage devices and while in transit over the network.
It is also concerned with scalability, therefore separates data cryptography from the server, leaving it to the clients; this approach is similar to the one used in CFS, and opposite to those taken on by other secure file system solutions (like NFS on top of IPsec).
FSFS is written as a pair of user space daemons that act as client and server. Because of this, it needs no kernel support (unlike NFS over IPsec), save the FUSE loadable kernel module on clients, included in Linux since 2.6.14; servers dont use FUSE and depend only on user space OpenSSL libraries.
Servers export an existing file system (of virtually any kind) to clients over the network through two separate channels: a TLS connection set up with OpenSSL, and a clear channel. Requests from the clients to the servers are sent via the TLS socket, thus they are encrypted and authenticated, according to TLS v1 specifications, by the channel itself and decrypted on receipt, as they are usually very short and the relevant cryptography does not constitute a great overhead; simple server replies undergo the same process.
Cryptography in this case happens at both ends of the transmission.
In a distributed file system, large amounts of data may be transferred between clients and servers, thus encrypting and decrypting everything may become too cumbersome for both parties, and as more clients are added to the system the server may severely lose performance; moreover, file data should be stored encrypted anyway, so the cryptography could be moved to the clients, in such a way that each encrypts data before a write operation sends it over the network to the server, and decrypts it after a read retrieves it.
This way servers only deal with TLS details and can concentrate on serving client requests by doing the relevant I/O on the underlying, "physical" file system. As the data is already encrypted, it does not need to go through the TLS channel and the corresponding overhead, but can be sent via the clear channel, provided the messages are authenticated.
Enhancements:
- This release fixes two bugs. One bug related to socket creation and would cause problems on some systems (namely OpenSUSE 10.2). The other bug related to server configuration creation when using the Python configuration utilities. Users dont need to upgrade to this release if theyre not experiencing problems or are not using the Python configuration utilities.
File Beamer 0.1.5
File Beamer is an easy to use file transfer tool. more>>
This is made possible by using Trolltechs Qt Library which provides an easy to use GUI toolkit, networking functions and a lot more.
File Beamer is free open source software and was released under the GPL license.
Usage:
Receiving files:
Click the "Receive" tab
If the receiver is behind a firewall or router, check "reverse connection" and enter the senders IP or host name. *
Click the "listen" button. The client can now receive files
When receving a file, a "Save as" dialog appears. Choose a place to safe the file.
Sending files:
Click the "Send" tab
If the receiver is behind a firewall or router check "reverse connection" *
Else input the target IP address or host name and port
Select a file to send either by typing the path or by clicking the "..." button
Click the "Send" button (target must be in listen mode)
File Splitter 1.3
Split large text/html files into smaller files. I find it much faster and more accurate than cut and paste. You embed commands in the big file telling it which pieces of it are to go where, then let Splitter do the work. It is much faster and more accurate than trying to select huge blocks of text in an editor. You dont accidentally lose or duplicate text. Keeping files small makes the site more responsive. more>>
File Splitter - Split large text/html files into smaller files. I find it
much faster and more accurate than cut and paste.
You embed commands in the big file telling it which pieces
of it are to go where, then let Splitter do the work. It is
much faster and more accurate than trying to select huge
blocks of text in an editor. You dont accidentally lose or
duplicate text. Keeping files small makes the site more
responsive.
In the following pretend that [...] are actually lessthan...greater than.
You embed multiple [split tags in the file to be split of
the form:
[split charlie.html]
...
stuff that will end up in the charlie.html file.
...
[/split]
The text between the [split xxx] and [/split] tags is split
off into that named file and the text is removed from the
original file along with the tags.
1. Filenames may be absolute or relative, with no quotes or spaces.
2. Tags may be nested, but they must balance (equal number
of [split xxx] and [/split]).
3. Tags are case-insensitive, i.e. may be lower or upper case.
4. Multiple [split xxx] tags may be directed to the same
file, where they will be appended.
5. If the files mentioned in the split tags already exist,
they will be overwritten.
6. Anything not inside [split xxx].. [/split] is retained in
the original file. Everything else is removed.
The file being split must be small enough to fit in RAM.
Java array addressing limits the file to 2GB, though other
considerations mean in practice the largest file you can
handle will be smaller still.
To install, Extract the zip download with Winzip, available from
http://www.winzip.com (or similar unzip utility) into any
directory you please, often C:\ -- ticking off the (user
folder names) option. To run as an application, type:
java -jar C:\com\mindprod\splitter\splitter.jar x.html
adjusting as necessary to account for where the jar file is.
Enhancements:
Version 1.3
allow you to specify encoding
System Requirements:<<less
KSendFile 0.8a
KSendfile is a native SAFT client. more>>
If a user A wants to send a file to user B they have the following choices:
- ftp to a recipients account B
In this case one has to know the password of the recipients account. If A and B are not the identical person, severe questions of security arise and it is not the method of choice. Even if account A and B belong to the same person, the typed ASCII password characters for ftp access will travel open (readable) as ordinary TCP packets through the network.
- ftp using an anonymous ftp server
The file has to be uploaded by A onto the anonymous ftp server using an ftp session. After that A has to notify B about this using e-mail informing this user about the fact that the file is ready for download now. B can get the file now by doing another ftp transfer. This procedures requires a common reachable ftp server, with an upload (write access) facility. During the time the file resides on the anonymous ftp server it is open to anyone, it may be readable, writeable, changeable or erasable.
- sending a file via e-mail
A is sending a file to B enclosed into an e-mail. According to RFC 822 a mail must contain characters from the NVT-ASCII character set, which is a subset of the 7 bit ASCII character set. The file transfer via e-mail is limited to English text documents or the text to be sent must be encoded properly, of course using the NVT-ASCII character set. Encoding procedures like uuencode or MIME are available. It is not a pleasure to use either of them, they dont support all file ownership attributes and they will enlarge the amount of data to be transferred by encoding. In addition to this most of the existing MTAs are not designed to handle large amount of data transfers.
To avoid all the described disadvantages the SAFT-protocol (Simple Asynchronous File Transfer) has been developed and the sendfile software has been created as a reference implementation for UNIX platforms.
Enhancements:
- Fixed a bug in ksendfilenetclient::sendFile when getting status code 531
- Added Resend Function
- Fixed bug in ksendfilenetclient::sendfile in the date function
- Now sends the file date, not the actual system date.
The Picture Transfer Protocol library 1.1.0
libptp2 is a library used to communicate with PTP devices like still imaging cameras or MP3 players (KODAK mc3). more>>
You can download files or tweak camera properties using ptpcam program boundled with libptp2.
Supported Cameras
The list of cameras implementing PTP is constantly growing. Most (if not all) USB still cameras designed after 2002 are supporting PTP. Below is the list of vendors implementing PTP in their cameras:
- Kodak
- HP
- Nikon
- Canon
- Panasonic
- Olympus
- Konica
- Minolta
- Fuji
- Ricoh
- Sony
More are comming...
Please note that if your camera is supporting PTP it does not mean that it is capable of performing all PTP operations (like trigerring capture, uploading files or tweaking properties). Sony cameras are good example of poor PTP implementation where the only thing you can do is downloading files. Sony even breaks the USB specification assigning all its cameras the same product/vendor IDs so dont buy Sony if you want to stay out of problems.
Currently most PTP features are supported. Uploading/downloading files, taking pictures and setting camera properties is supported as long as your camera supports it. However there are many Vendor Extensions that are not supported.
For example Canon uses some proprietary extension operations to take pictures (entering capture mode, extending lens and so on), Nikon introduces proprietary PTP properties.
Unfortunately I have access to couple of Kodak cameras only so to help support extension features you may send me a patch or donate/lend the camera.
Enhancements:
- A number of bugs were fixed, including leaving the camera in an unpredictable state when an I/O error occurs.
- The projects own reimplementation of the old libusb interface (IOCTL_USB_BULK) is used instead of the new URB interface, which is 20% to 50% slower and sometime leads to I/O errors.
- File time preservation across download was added.
- Better capture support was added.
- A bug that caused files over 2MB to be corrupted while downloading was fixed.
- The --loop-capture feature was added.
- Configure errors and parralel build with make -j were fixed.
Large Database Backup 1.0
Large Database Backup is a PHP class can be used to backup large MySQL databases into multiple files. more>>
It can list the tables of a given MySQL database and generate SQL statements to create and insert records with the current values of the table rows, as if it was creating a new database. The class stores the generated SQL statements in files.
A full backup process can be split in multiple iterations to not exceed PHP default script execution time limit. Each iteration dumps a limited number of table rows. An iteration can be executed by a script that redirects the page request to itself at the end to proceed to the next iteration.
FileSystemStorage 2.5.2
FileSystemStorage is a project that provides FileSystem storage for Archetypes. more>>
FileSystemStorage is an Archetype storage for storing data on FileSystem This storage is used to avoid ZODB to grow quickly when using a lot of large files.
SSH Filesystem 1.8
SSH Filesystem is a filesystem client based on the SSH File Transfer Protocol. more>>
The idea of sshfs was taken from the SSHFS filesystem distributed with LUFS, which I found very useful. There were some limitations of that codebase, so I rewrote it.
Main features:
- Based on FUSE (the best userspace filesystem framework for linux ;-)
- Multithreading: more than one request can be on its way to the server
- Allowing large reads (max 64k)
- Caching directory contents
Installing
First you need to download FUSE 2.2 or later from http://fuse.sourceforge.net/. You also need to install the devel package for glib2.0. After installing FUSE, compile sshfs the usual way:
./configure
make
make install (as root)
How to mount a filesystem
Once sshfs is installed (see next section) running it is very simple:
sshfs hostname: /mountpoint
Note, that its recommended to run it as user, not as root. For this to work the mountpoint must be owned by the user. If the username is different on the host you are connecting to, then use the "username@host:" form. If you need to enter a password sshfs will ask for it (actually it just runs ssh which ask for the password if needed). You can also specify a directory after the ":". The default is the home directory.
Enhancements:
- This release fixes a couple of rarely occurring crashes and some smaller bugs.