Unable to execute ./start_logs to test flume for CCA175 prep

flume
#1

Hi,

Unable to execute ./start_logs to test flume for data injestion,even i copied the script at my /home/tarunkumard/flume folder still saying permission issue

0 Likes

#2

Hey - did you try creating logs/access.log in your home directory ? I created logs directory and copied access.log to it and then ran start_logs and it worked fine.

Regards,
Shanil

1 Like

#3

You wud need to edit the script as well , change the path of access.log - point it to your home directory logs/access.logs … Sorry for the staggered reply

Regards,
Shanil

0 Likes

#4

Hi Tarun,

If you are using itversity cluster, you do not need to do any configuration for running start_logs.sh. You just need to execute start_logs.sh and the log file will be generated in /opt/gen_logs/logs folder.
But now it seems that the start_logs.sh file present in modified with log location changed to /home/tarunkumard/logs/access.log.
You need to change the following line in start _logs.sh
lib/genhttplogs.py > /home/tarunkumard/logs/access.log &
to
lib/genhttplogs.py > logs/access.log &

It seems you have made these changes and consequently no one will be able to use start_logs.sh on the cluster since they will not have access to /home/tarunkumard/logs folder.

Thanks,
Pratyush

0 Likes

#5

reverted as mentioned ,but your above “You just need to execute start_logs.sh and the log file will be generated in /opt/gen_logs/logs folder.” that is not working which is what i mentioned above …please revert in case of any questions

0 Likes

#6

Hi Pratyush,

this is not working for me either.

If I try to execute the script with ./start_logs I am getting the message:

./start_logs.sh: line 4: logs/access.log: Permission denied

Thanks,
Matthias

0 Likes

#7

@itversity I am also getting the same problem

[cjoseph48@gw02 gen_logs]$ start_logs.sh
/usr/bin/start_logs.sh: line 4: logs/access.log: Permission denied

kindly advice
@Matthias_Schmidt are you able to resolve the above issue?

Regards
Joseph

0 Likes

#8

No, unfortunately not. However, I created the same files in in my personal folder (NOT copying files BUT ONLY the content, for some reason I received a permission error again when I just copy the files). Now, it works for me.

Now, I am facing issue when running flume:
@itversity: There is no output generated on the console. Instead, I am getting:

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property ‘log4j2.debug’ to show Log4j2 internal initialization logging.

So I have added --conf /etc/flume/conf/

Which results in:

ERROR RollingFileManager (/tmp/flume/flume.log) java.io.FileNotFoundException: /tmp/flume/flume.log (Permission denied) java.io.FileNotFoundException: /tmp/flume/flume.log (Permission denied)

Please assist.

Thanks,
Matthias

0 Likes

#9

@Matthias_Schmidt I have raised ticket with itversity support. let us wait.

Regards
Joseph

0 Likes

#10

Try to creat new conf file in your floder and run command like below command

flume-ng agent --name wh --conf-file your-conf-file-name --conf /etc/flume/conf

0 Likes

#11

Hi, do I need to create the .sh files separately for my id?

As I am unable to run the start_logs.sh

Thanks
Joseph

0 Likes

#12

You don’t need to run the start_logs.sh, we will be running that from admin side. But there is some issue, we will be looking into this.

We will update once it is resolved.

0 Likes

#13

Kindly resolve, still I am getting the same issue.

Regards
Joseph

0 Likes

#14

@Joseph - you can use the tail_logs.sh to see the logs. Please find the screenshot for reference. I mailed you that we already started start_logs.sh from our side.

0 Likes

#15

Thank you!!
@vinodnerella. I am able to see the logs.

@Matthias_Schmidt

Regards
Joseph

0 Likes

#16

Thanks a lot.

But the does not solve the issue which I described above.

@vinodnerella: Can you please assist here as well?

@Joseph: Are you facing the same issue when running flume?

Best regards,
Matthias

0 Likes