FileZilla Forums

Welcome to the official discussion forums for FileZilla
Donate to project
It is currently 2014-04-23 06:39

All times are UTC




Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2, 3  Next
Author Message
PostPosted: 2010-01-27 04:29 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
Ey fellas, some of you may have seen me snooping around here for awhile promising that I was creating a log parser that would make all of our lives easier. Well I finally released it minutes ago on sourceforge.net. You can find it at this link: http://sourceforge.net/projects/fla

It is in alpha mode as there are a number of features I would like to add to it, not so much because it is buggy(although it may have a few). I have extensively tested it(partly the reason I took so long to release it) and I am confident that there are no major bugs in it. To run it you will have to use terminal/command-line as there currently is no GUI, and it will produce text files with the interpreted data. Be sure to read the README file before using it as there is a good amount of stuff the script can do.

So be sure to check it out, and feel free to give me feedback on it. And if you are interested in helping with the project please let me know as I am always looking for people to help. If you have trouble with it at all or have bugs to report please contact me(my contact info can be found in the README with the project). Anyways I hope this helps with managing your server and makes your life a little easier. Enjoy!

List of all release announcements:
1.03 alpha: http://forum.filezilla-project.org/viewtopic.php?p=68383#p68383
1.03a alpha: http://forum.filezilla-project.org/viewtopic.php?p=69201#p69201
1.10 alpha: http://forum.filezilla-project.org/viewtopic.php?p=69896#p69896


Last edited by Stunner on 2010-09-28 22:16, edited 3 times in total.

Top
 Profile  
 
PostPosted: 2010-01-27 06:46 
Offline
Site Admin
User avatar

Joined: 2004-02-23 20:49
Posts: 22551
Thanks, looks promising.


Top
 Profile  
 
PostPosted: 2010-02-19 08:23 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
I have noticed that there are quite a few posts with portions of server logs displayed and I am assuming most of the posters of those log files don't bother changing the IP addresses in those logs before posting them on the forum. I just wanted to make people aware that the FileZilla Log Analyzer can also be used to mask sensitive information by recreating your log file with scrambled user names, files/folders, and IP addresses. So I would highly recommend using that feature rather than blatantly revealing your user's(and your own) IP addresses for everyone on the net to see. And as always, feedback is appreciated! :)


Top
 Profile  
 
PostPosted: 2010-02-19 10:27 
Offline
226 Transfer OK
User avatar

Joined: 2006-05-01 03:28
Posts: 19659
Location: Germany
Please note that in most cases posting logs with obscured IPs makes them useless from a support perspective. If you change all occurrences of the IP to another, that's OK, as long as you maintain consistency and regard the difference LAN IP vs. WAN IP.

_________________
### BEGIN SIGNATURE BLOCK ###
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
All support requests per PM will be ignored!
### END SIGNATURE BLOCK ###


Top
 Profile  
 
PostPosted: 2010-02-19 10:46 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
What do you mean by the "difference [between] LAN IP vs. WAN IP"? And yes, it does maintain consistency. It works this way: Upon first encounter of a new IP address it creates a random IP address and stores it in a dictionary with the original authentic IP address being used as a key. So whenever the real IP address is encountered again the same "fake" IP address is used instead. The same goes for user names and files/folders.


Top
 Profile  
 
PostPosted: 2010-02-19 13:05 
Offline
226 Transfer OK
User avatar

Joined: 2006-05-01 03:28
Posts: 19659
Location: Germany
Quote:
What do you mean by the "difference [between] LAN IP vs. WAN IP"?
Let's say there's a 192.168.0.1 in the log. If now your tool randomizes this IP to, say 123.234.432.321, we would now incorrectly assume the IP is public, while in fact it's not. So you would destroy information. Of course there's the other way around, if 66.54.208.99 would be randomized to 10.58.209.79, then it suddenly would be a LAN IP, again giving us the wrong impression. (All IPs are examples).

So your tool MUST exclude the private IP ranges from any randomization.
  • Class A - 10/8
  • Class B - 172.16/12
  • Class C - 192.168/16
  • Link local - 169.254/16
  • Loopback - 127/8

  • Loopback IPv6 - ::1/128
  • Link local IPv6 - fe80::/10
  • Site Local Unicast IPv6 - fec0::/10
  • Unique Local Unicast IPv6 - fc00::/7

All IPs in (shorted) CIDR notification. The list may not be complete.

If the source IP is in these ranges, don't touch it. If the result of a randomization is in these ranges, take a new seed and repeat the operation.

Private addresses are not unique and can't be routed over the internet. Thus they don't need "protection".

_________________
### BEGIN SIGNATURE BLOCK ###
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
All support requests per PM will be ignored!
### END SIGNATURE BLOCK ###


Top
 Profile  
 
PostPosted: 2010-02-19 16:44 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
Ah ok, interesting. I will be sure to fix that in the next update. Where can I find a complete list of these IP addresses? And thanks for pointing this out to me.


Top
 Profile  
 
PostPosted: 2010-02-19 17:14 
Offline
226 Transfer OK
User avatar

Joined: 2006-05-01 03:28
Posts: 19659
Location: Germany
I got some from http://en.wikipedia.org/wiki/Private_network, but the complete list should be at http://www.iana.org/numbers/ (can't get more official :wink:).

IPv4: http://tools.ietf.org/html/rfc3330
IPv6: http://tools.ietf.org/html/rfc5156

_________________
### BEGIN SIGNATURE BLOCK ###
FTP connection problems? Do yourself a favor and read Network Configuration.
All FileZilla products fully support IPv6. http://worldipv6launch.org
All support requests per PM will be ignored!
### END SIGNATURE BLOCK ###


Top
 Profile  
 
PostPosted: 2010-02-19 18:45 
Offline
Site Admin
User avatar

Joined: 2004-02-23 20:49
Posts: 22551
See also IP_address


Top
 Profile  
 
PostPosted: 2010-08-19 10:44 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
So I am about to finally embark on this now that I have some free time and was wondering what I should do in the case of IPv6 as the logs from FileZilla Server are all in the IPv4 format(unless I am missing something). So I would only enforce those restrictions for IPv4 and completely ignore those for IPv6, correct? Thanks.


Top
 Profile  
 
PostPosted: 2010-08-19 20:23 
Offline
Site Admin
User avatar

Joined: 2004-02-23 20:49
Posts: 22551
Right now FileZilla Server does not even support IPv6 at all.


Top
 Profile  
 
PostPosted: 2010-08-23 13:24 
Offline
504 Command not implemented

Joined: 2010-08-23 13:20
Posts: 11
uhm there any version with a GUI in the planning that dont involve manual typing of all commands?
:roll:

cant get this manual version to work (copied log1.log to the install dir where FLA.py is located)

D:\parcer\FileZilla Log Analyzer-alpha1.02>python FLA.py -d log1.log
Analyzing log1.log...
Traceback (most recent call last):
File "FLA.py", line 1726, in <module>
main(sys.argv)
File "FLA.py", line 1649, in main
(m,p,d,t,u,i,a) = FLAmodule.getLine(line)
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 114, in getLine
return _parse(line)
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 72, in _parse
u = _parseUsr(6,fline) #user
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 38, in _parseUsr
if fline[i][0] == '(': #if next element is ip address
IndexError: list index out of range

D:\parcer\FileZilla Log Analyzer-alpha1.02>


Top
 Profile  
 
PostPosted: 2010-08-23 21:53 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
Legs wrote:
uhm there any version with a GUI in the planning that dont involve manual typing of all commands?
:roll:

cant get this manual version to work (copied log1.log to the install dir where FLA.py is located)

D:\parcer\FileZilla Log Analyzer-alpha1.02>python FLA.py -d log1.log
Analyzing log1.log...
Traceback (most recent call last):
File "FLA.py", line 1726, in <module>
main(sys.argv)
File "FLA.py", line 1649, in main
(m,p,d,t,u,i,a) = FLAmodule.getLine(line)
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 114, in getLine
return _parse(line)
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 72, in _parse
u = _parseUsr(6,fline) #user
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 42, in _parseUsr
return ' '.join((fline[i-1],_parseUsr(i+1,fline)))
File "D:\parcer\FileZilla Log Analyzer-alpha1.02\FLAmodule.py", line 38, in _parseUsr
if fline[i][0] == '(': #if next element is ip address
IndexError: list index out of range

D:\parcer\FileZilla Log Analyzer-alpha1.02>


Unfortunately no current plans for a GUI version, but I have definitely been thinking about it :). What I need to implement is more functionality first, such as the ability for the analyzer to take logs piece by piece and append it to the summary as opposed to requiring everything to be provided all at once.

Thanks for bringing the problem to my attention, but I can't really pinpoint the problem without looking at the log file/running some tests of my own. If you wouldn't mind sending the log file you are using to me via email(my email address can be found in the README) that would be great. It is a parsing error so it might be a case that I am not checking for, or something you have inadvertently changed in the log file.


Top
 Profile  
 
PostPosted: 2010-08-25 15:12 
Offline
504 Command not implemented

Joined: 2010-08-23 13:20
Posts: 11
sure thing. I edited out some personal information (replaced them with bogus numbers) and tested it again and same problem as before so its not corrupt.
log attached


Attachments:
log1.log [52.32 KiB]
Downloaded 457 times
Top
 Profile  
 
PostPosted: 2010-08-26 01:35 
Offline
226 Transfer OK

Joined: 2009-03-31 01:19
Posts: 61
Legs wrote:
sure thing. I edited out some personal information (replaced them with bogus numbers) and tested it again and same problem as before so its not corrupt.
log attached


Really appreciate it. I fixed your issue but stumbled upon another which I am going to attempt to fix before making another release. I will post on this thread when I release the new version(should be fairly soon).


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 40 posts ]  Go to page 1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 12 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Dedicated server provided by Artmotion.
Forum sponsored by Everyware.ch.
Powered by phpBB® Forum Software © phpBB Group